-- Logs begin at Wed 2022-07-27 19:22:43 UTC, end at Wed 2022-07-27 21:08:04 UTC. -- Jul 27 19:53:53.615666 dv-cinder-799871-44-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 19:53:53.624105 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 19:53:53 2022] *** Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: nodename: dv-cinder-799871-44-iscsi Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: machine: x86_64 Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: clock source: unix Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: pcre jit disabled Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: detected number of CPU cores: 4 Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: current working directory: / Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: detected binary path: /usr/bin/uwsgi-core Jul 27 19:53:53.626132 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: your processes number limit is 47680 Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: your memory page size is 4096 bytes Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: detected max file descriptor number: 2048 Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: lock engine: pthread robust mutexes Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: thunder lock: enabled Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 19:53:53.626720 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 19:53:53.665403 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: Python main interpreter initialized at 0x55a7386069c0 Jul 27 19:53:53.665403 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: python threads support enabled Jul 27 19:53:53.665403 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: your server socket listen backlog is limited to 100 connections Jul 27 19:53:53.665403 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: your mercy for graceful operations on workers is 90 seconds Jul 27 19:53:53.665748 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 19:53:53.665825 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: *** Operational MODE: preforking *** Jul 27 19:53:53.666095 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: *** uWSGI is running in multiple interpreter mode *** Jul 27 19:53:53.666095 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: spawned uWSGI master process (pid: 80129) Jul 27 19:53:53.666294 dv-cinder-799871-44-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 19:53:53.673081 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: spawned uWSGI worker 1 (pid: 80136, cores: 1) Jul 27 19:53:53.673081 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: spawned uWSGI worker 2 (pid: 80137, cores: 1) Jul 27 19:53:53.673081 dv-cinder-799871-44-iscsi devstack@keystone.service[80129]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 19:53:59.181127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:59.181127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: from cryptography.utils import int_from_bytes Jul 27 19:53:59.181897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:59.181897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: from cryptography.utils import int_from_bytes Jul 27 19:53:59.181897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:59.181897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: from cryptography.utils import int_from_bytes Jul 27 19:53:59.182386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 19:53:59.182386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: from cryptography.utils import int_from_bytes Jul 27 19:53:59.232035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 19:53:59.232219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 19:53:59.232306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 19:53:59.232398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 19:53:59.232462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] command line args: [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 19:53:59.232541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] command line args: [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 19:53:59.232661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 19:53:59.232749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 19:53:59.232848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ================================================================================ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 19:53:59.232930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ================================================================================ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 19:53:59.233142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] admin_token = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] admin_token = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_dir = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_dir = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_source = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_source = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.233925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] debug = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] debug = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.234950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_config_append = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_config_append = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.235947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_dir = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_dir = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.236976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.237927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.238941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.239936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] publish_errors = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] publish_errors = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.240989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] standard_threads = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] standard_threads = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.241985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_journal = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_json = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_journal = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_stderr = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_json = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_syslog = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_stderr = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.242952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.243028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_syslog = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.243163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.243244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 19:53:59.243400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.243514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.243656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.243735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.243891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.244915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.245955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.246875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.247976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.248797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.249982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.250925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.251603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.252834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.253932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.254943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.255895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.258878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.259965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.260898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.261954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.262879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263495 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.263986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.264911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.265932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.266848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.267001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.267078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.267234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.267351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.268875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.external = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.external = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.password = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.269877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.token = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.password = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.token = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.270950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.271971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.272954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.273959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.274957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.275961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.276914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.277874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.278902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.279919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.280998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.281916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.282939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.283543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.283651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.283730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.283803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.283884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.284908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.285896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.286997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.287998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.288938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.289920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.290981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.291926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.292946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.293949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.294996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.295893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.296905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.297928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.298832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.299883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.300915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.301979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.302944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.303873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.304897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.305872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.306897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.307936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.308949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.309877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.310902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.311906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.312918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.313937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.314073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.314762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.314895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.315742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.316906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.317948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.318944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.319937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.320945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.321993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 19:53:59.322638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 19:53:59.322638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 19:53:59.348945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.349168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.349352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.349581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.349816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.350044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.357670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.358345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.388155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.388382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.388851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.389119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.443208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.443483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.493657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.493890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.494796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.495031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.558614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.558837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.558977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.559932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.560063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.560192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.583566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.583794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.583941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.584860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.585010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.585138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.585269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.585269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.595967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.596419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.608691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.608883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.609893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.610684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 19:53:59.627920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.628120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:59.628281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.628454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.628596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:59.628728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:59.628863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 19:53:59.629282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.629918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.630051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.639360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.639637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.639766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.639903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.640031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.640168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.640297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.640451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.647161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.647436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.651872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.652112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.652245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.652386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.652519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.652661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.652789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.652920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.653982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.654748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.678431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.678582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.679913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.680055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.680194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.680322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.684816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.684971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.685127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.685256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.685392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.685523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.692440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.692709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.692848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.693012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.696851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.697157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.697285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.697444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.697583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.697709 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.697836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.697962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.698865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.699060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.715318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.715601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.715815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.716090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.719531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.720041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.723296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.723588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.724005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.724298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.727990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.728697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.731261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.731908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.734459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.734751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.735971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.736791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.737121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.752574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.752857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.753148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.753382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.753480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.753664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.753929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.754190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.760280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.761128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.761853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.762123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.762692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.762965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.764918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.765216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.765505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.765796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.765879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.766081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.766173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.766369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.766670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.766957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.778339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.778621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.778945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.779224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.779850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.780046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.780143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.780320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.780594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.796693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.797893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.799935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.800900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.802913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.803885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.805430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.806444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.806988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.807270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.807548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.807963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.808256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.808563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.814045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.816050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.816585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.817701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.819542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.819829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.820106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.820375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.820639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.820927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.821237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.821514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.828468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.828751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.829050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.829646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.830475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.830767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.836701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.836987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.837271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.837554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.837687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 19:53:59.837833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.837969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.838803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.839962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.840113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.840246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.840371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.840525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.840789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.841088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.841350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 19:53:59.873936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a7386069c0 pid: 80136 (default app) Jul 27 19:53:59.874762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a7386069c0 pid: 80137 (default app) Jul 27 19:53:59.877764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71341-e393-492b-8cbc-d4a1c94d2fca None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:53:59.878001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71341-e393-492b-8cbc-d4a1c94d2fca None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:53:59.878274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f71341-e393-492b-8cbc-d4a1c94d2fca None None] PATH_INFO: `/v3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:53:59.882680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Wed Jul 27 19:53:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:07.526943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.527500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.527500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.531972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:07.537834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:07.538539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:07.538767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:07.610097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: INFO dbcounter [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] Registered counter for database keystone Jul 27 19:54:07.631515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG oslo_db.sqlalchemy.engines [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80136) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 19:54:07.637848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writer thread running {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 19:54:07.815791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG passlib.handlers.bcrypt [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80136) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 19:54:07.816383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG passlib.handlers.bcrypt [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80136) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 19:54:07.870830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:07.872440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:07.995083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 27 19:54:08.154211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:08.224206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: INFO dbcounter [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] Registered counter for database keystone Jul 27 19:54:08.243232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG oslo_db.sqlalchemy.engines [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80137) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 19:54:08.249321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writer thread running {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 19:54:08.366282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e338f7c-3202-4b6b-99e8-acc7f0ff6d49 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:08.370667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:08.371314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:08.371599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:08.371877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:08.380356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:08.431539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING keystone.common.rbac_enforcer.enforcer [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 19:54:08.431884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5dba6-23e4-4bb2-9bef-1a7786a34770 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:08.442498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Jul 27 19:54:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 19:54:08.448473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:08.463788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1c944-95f3-45ff-9843-6ee7e7b78e4d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:08.468315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:08.469456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:08.469746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:08.470023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] PATH_INFO: `/v3/projects/9e55cde5d6044829b509ee6bccad61b8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:08.471740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] RBAC: Authorizing `identity:get_project(project_id=9e55cde5d6044829b509ee6bccad61b8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:08.527285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 19:54:08.527631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0676f-ef44-4661-a1df-df0fa3f3b7b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:08.532297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Wed Jul 27 19:54:08 2022] GET /identity/v3/projects/9e55cde5d6044829b509ee6bccad61b8 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:10.511329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a2e02-36ec-454a-8ae7-3c588b8db928 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.511868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a2e02-36ec-454a-8ae7-3c588b8db928 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.511868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a2e02-36ec-454a-8ae7-3c588b8db928 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.515989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:10.521515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.521915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.522080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.542177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:10.543683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:10.552092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 19:54:10.573720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a2e02-36ec-454a-8ae7-3c588b8db928 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:10.585970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a2e02-36ec-454a-8ae7-3c588b8db928 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:10.590157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:10.590759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.591043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.591323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.599714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:10.601541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaad7bf-1e83-4572-bd64-ca17f1b7b6e7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:10.610269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Jul 27 19:54:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 19:54:10.616174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:10.625132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d8b59-4eb3-47e3-8aa8-0bb6006e6ef7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:10.629652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:10.630274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:10.630557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:10.630830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:10.632180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:10.634036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4fcdc-d56d-4bfd-a870-35b0a0d4b912 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:10.635565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Wed Jul 27 19:54:10 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:12.666167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0d24-c220-4f1a-af06-7e72018c46c2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.666685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0d24-c220-4f1a-af06-7e72018c46c2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.666785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a0d24-c220-4f1a-af06-7e72018c46c2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.668038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:12.673089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2e878-1f79-4f21-b4c8-297a570becc7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.673476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2e878-1f79-4f21-b4c8-297a570becc7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.673820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2e878-1f79-4f21-b4c8-297a570becc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.693580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bcb2e878-1f79-4f21-b4c8-297a570becc7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:12.694765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bcb2e878-1f79-4f21-b4c8-297a570becc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:12.774508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 6/11] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:12.776627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb10a6-19f7-4451-bf90-0e13956a8e88 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.776993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb10a6-19f7-4451-bf90-0e13956a8e88 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.777366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb10a6-19f7-4451-bf90-0e13956a8e88 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.778787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 6/12] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:12.784170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.784466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.784752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.802428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:12.803234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb10a6-19f7-4451-bf90-0e13956a8e88 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.803636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:12.821172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:12.874175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 7/13] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:12.895699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:12.951873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb10a6-19f7-4451-bf90-0e13956a8e88 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.985174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:12.986180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:12.986467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:12.986864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:12.995660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274af424-ab23-4bba-820e-1f3f34be9cd7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:12.998037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:13.007266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:13.008335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:13.009181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 7/14] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:13.014480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.029815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.030438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.030718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.030990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.049947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.052320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.099992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.101519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.122365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9458d-2543-442e-8da0-81799a78c8ea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.155434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.156050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.156336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.156616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.157376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.157688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.159646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.171091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 8/15] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:13.177096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.201138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 8/16] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:13.207290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.256595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.269353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.275660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe510178-f87c-4a99-a89e-e9b3dc1d4cc4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.304618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9a352-7dd9-43f4-9a24-77f5f0019536 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.309641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.310288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.310574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.310857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.312474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.314629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.316722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 9/17] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:13.322278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.340495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175759-b753-4824-893b-e1b79506b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.341176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175759-b753-4824-893b-e1b79506b983 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.341461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175759-b753-4824-893b-e1b79506b983 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.341739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35175759-b753-4824-893b-e1b79506b983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.351773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35175759-b753-4824-893b-e1b79506b983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.353277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35175759-b753-4824-893b-e1b79506b983 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.427699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728fef-e5b3-48c3-8d39-aa8dc7255c4d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.463699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.464270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.464556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.464836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.466149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.467611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.469235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 10/18] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-35175759-b753-4824-893b-e1b79506b983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:54:13.477114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:13.477994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:13.478781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:54:13.478781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:54:13.478781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:13.478781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:13.478781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 9/19] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:13.480100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.485553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175759-b753-4824-893b-e1b79506b983 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:13.513992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.581366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c76a3-c1d8-4492-9348-13487ed1abda None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.583069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35175759-b753-4824-893b-e1b79506b983 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:13.585365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:13.617121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.618133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.618612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.619080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.620115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:13.620452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.domain.id=default, user.id=8e99a25baa6a4ee4875a0277c6006aec) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.620990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.621094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:13.621543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:13.621986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:13.623114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:13.623514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b2a64-cb5f-4f4f-834a-3578ca1091a8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.623610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:13.626050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8b520-e939-418c-9e9b-e4c3ce676560 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:13.649095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 11/20] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/role_assignments?user.id=8e99a25baa6a4ee4875a0277c6006aec&scope.domain.id=default&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:13.649300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 10/21] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:15.882412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1cc9-2c0a-4333-8759-2044b11574ad None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.883437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1cc9-2c0a-4333-8759-2044b11574ad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.883917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e1cc9-2c0a-4333-8759-2044b11574ad None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.885819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25b24-fbce-42b7-a328-3982ea6eaed2 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.886570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25b24-fbce-42b7-a328-3982ea6eaed2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.887056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25b24-fbce-42b7-a328-3982ea6eaed2 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.890266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 11/22] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:15.890665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 12/23] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:15.895632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.896154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.896656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.898056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:15.898534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:15.898995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:15.939558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG passlib.handlers.bcrypt [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80137) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 19:54:15.941534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG passlib.handlers.bcrypt [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80137) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 19:54:15.942662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:15.950202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:16.020489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:16.022461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:16.138540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 12/24] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:16.166840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 13/25] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:16.189949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.202348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.267768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.377195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.391147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b91657-4fe5-4e9f-9e4d-fd07cdd6d7d5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.413771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc00fd-f7d0-41c7-9a98-89d78e4e051e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.467598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.468272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.468561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.468841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.492481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.493416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.493705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.493982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.495252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:16.502863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:16.503756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:16.504904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 13/26] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:16.512179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.531767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.533845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:54:16.561619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:16.562466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:16.563703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:54:16.563703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:54:16.563703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:54:16.563703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:16.563703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 14/27] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:16.569681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.658766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035cd5-069b-4e86-87d7-1efa31ac2a52 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.713855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.714512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.714804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.715080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.715819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:16.716125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.717578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a88e0-a217-413f-a570-13e43db565d3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.749608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.750937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432ee4-5b18-4892-b7ad-5768ad7d4138 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.751165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432ee4-5b18-4892-b7ad-5768ad7d4138 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.751438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1432ee4-5b18-4892-b7ad-5768ad7d4138 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.752711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 15/29] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:16.756459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432ee4-5b18-4892-b7ad-5768ad7d4138 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.799771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.869844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:16.898734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8575d-f7fc-40bf-bd51-b9e074d64b55 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.920396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1432ee4-5b18-4892-b7ad-5768ad7d4138 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:16.944751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.945403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.945692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.946363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.947081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:16.947545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.949426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb195a-4bc4-42bc-8fa8-86c45761534e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.958899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:16.959965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.960559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.961080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.962545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:16.964313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dabffc0-a01d-4ee2-a3aa-cfd65a4e7a52 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:16.966590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 16/30] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:16.968115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.968594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.969059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.970749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 15/31] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:16.972692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b90eb-03bf-4be1-8c3f-a676439686fe None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:16.973247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b90eb-03bf-4be1-8c3f-a676439686fe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:16.973660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b90eb-03bf-4be1-8c3f-a676439686fe None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:16.975390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 16/32] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:16.976236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b90eb-03bf-4be1-8c3f-a676439686fe None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:16.994147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:16.995869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.074990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 17/33] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:17.076200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.103107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.121110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b90eb-03bf-4be1-8c3f-a676439686fe None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.165647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.167390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.167815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.168225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.169739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.171427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7aa8d-436d-481f-aef8-97a58b2a20bd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.175088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 17/34] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:17.176949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.177662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.178079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.195646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.202241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292cc133-97bd-4d32-8494-fd186876b52c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.202461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:17.203812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.242445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.243215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.243745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.244195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.245044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.246681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.288019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 18/35] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:17.289049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.397246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 18/36] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/projects => generated 301 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:17.405229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba0e1a-cdab-48fa-9745-b08aed883eda None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.405431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.446809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.447620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.447957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.448681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.449751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.452011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.462103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.503748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.548000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1402e-5d14-4626-a3ec-75fed82a6ec0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.587345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.588089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.588384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.588667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] PATH_INFO: `/v3/domains/default/users/8e99a25baa6a4ee4875a0277c6006aec/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.590995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8e99a25baa6a4ee4875a0277c6006aec, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.592600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722b603-443e-4b88-a596-a19211685bd5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.593562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 19/37] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/roles => generated 225 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:17.594352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.614929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 19/38] 10.222.0.65 () {58 vars in 1361 bytes} [Wed Jul 27 19:54:17 2022] PUT /identity/v3/domains/default/users/8e99a25baa6a4ee4875a0277c6006aec/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:17.617672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1afb-558f-44a8-9911-f2101e607a58 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.618280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1afb-558f-44a8-9911-f2101e607a58 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.618794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed1afb-558f-44a8-9911-f2101e607a58 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.620562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 20/39] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:17.622163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587dff4-8bbe-449e-a2d1-10594527bb52 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.622708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587dff4-8bbe-449e-a2d1-10594527bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.623185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587dff4-8bbe-449e-a2d1-10594527bb52 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.625186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 21/40] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:17.626773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063acac-216a-438f-962d-fa01679dd57e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.627275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063acac-216a-438f-962d-fa01679dd57e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.627808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063acac-216a-438f-962d-fa01679dd57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.662354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c063acac-216a-438f-962d-fa01679dd57e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:17.665093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c063acac-216a-438f-962d-fa01679dd57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.738958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8952021-a6c1-4d42-8766-2b17c6aec195 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:17.779597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 22/41] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:17.781810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.782408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.782699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.802199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:17.802910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:17.803196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:17.803477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:17.804414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:17.806128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:17.806545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:17.807761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:17.828702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 20/42] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/roles => generated 219 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:17.829511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.892561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.916838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 23/43] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:17.922380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:17.946632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:17.988507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127de69d-d044-4806-8368-e91cd7dee6d6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.040359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.040908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.041223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.041500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.042215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.043616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.062820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 21/44] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Jul 27 19:54:17 2022] POST /identity/v3/roles => generated 223 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:18.080395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fdc9-7854-4ebd-a908-6569d1e88745 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.123636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.124658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.125166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.125625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.127743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.129539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fcf5b-33e4-4274-a001-dcd32f61b12a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.131550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 24/45] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:18.137846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:18.256378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400c690-60b9-4549-b7c7-fc85b54a69d3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:18.274212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:18.293717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:18.294288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:18.294572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:18.294848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:18.295572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:18.297048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934d80e-72c2-42ba-8711-3cc997b1c4e6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:18.325772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 22/46] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:18 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:19.738345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edca8e-5780-49e9-9052-f61fabf76256 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.738859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edca8e-5780-49e9-9052-f61fabf76256 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.738941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edca8e-5780-49e9-9052-f61fabf76256 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.740246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 25/47] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:19.745494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.745907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.746056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:19.765327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:19.766841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:19.835354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 23/48] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:19.857369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edca8e-5780-49e9-9052-f61fabf76256 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:19.915800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:19.960940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edca8e-5780-49e9-9052-f61fabf76256 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:19.995604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:19.996170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:19.996455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:19.996733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.007227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:20.012258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:20.013102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:20.018942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 26/49] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:20.021112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.119685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.123766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b5ba-bdd4-4a7a-b5b2-baaaf477d3ff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.157563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.158739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.159024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.159301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.160019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:20.160328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.161741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.169760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 24/50] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:20.175378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.276194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010ac59-3034-4720-823c-878629ffa6c4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.290756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.311051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.311700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.311987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.312266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.313476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.315496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158ae5-a174-460e-9f10-e58362b9c35a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.317395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 27/51] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:20.322827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.331163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58e9b-7f22-4281-bd55-509b41ee5192 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.331451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58e9b-7f22-4281-bd55-509b41ee5192 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.331734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58e9b-7f22-4281-bd55-509b41ee5192 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.333089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 28/52] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:20.338032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.338323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.338609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.355932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:20.357185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:20.424110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 29/53] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:20.427557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc99357-012a-408d-939f-1a1557088519 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.446735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.460733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.461411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.461698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.461978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.463126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.464589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.466232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 25/54] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:20.472337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.500580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.543692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d935ea4-be7c-437f-8638-841119e06342 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.556680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:20.571720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b629e-c570-4f40-bd2c-53adc724b516 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.577517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.578157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.578453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.578727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.580409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.581904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.583501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 30/55] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:20.589037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:20.606884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.607511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.607795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.608074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.608864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.domain.id=default, user.id=8e99a25baa6a4ee4875a0277c6006aec) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:20.609208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.610660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8ab0e-8b96-44ff-a8ba-94e0efa39ee2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.617094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 26/56] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Jul 27 19:54:20 2022] GET /identity/v3/role_assignments?user.id=8e99a25baa6a4ee4875a0277c6006aec&scope.domain.id=default&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:20.702761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c60422-628e-4d7d-b8fe-36b014457d27 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:20.741601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:20.742290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:20.742508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:20.742785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:20.743489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:20.744990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530c6d9-b639-42b1-9f7e-4a5d9cee7633 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:20.773003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 31/57] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Jul 27 19:54:20 2022] POST /identity/v3/projects => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:22.962426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4716-8acd-4e97-aed2-5e6afd744755 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.962937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4716-8acd-4e97-aed2-5e6afd744755 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.963024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b4716-8acd-4e97-aed2-5e6afd744755 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.964334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 27/58] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:22.969672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:22.970062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:22.970237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:22.988392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:22.989623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:23.014162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.054593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 32/59] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:23.078814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b4716-8acd-4e97-aed2-5e6afd744755 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.170212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:23.180230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b4716-8acd-4e97-aed2-5e6afd744755 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.214167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:23.214753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.215035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.215315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.216458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:23.217953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9dd74-5e21-45e3-8a0b-d5fe21efc1cb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:23.219526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 28/60] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:23.225355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:23.320167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2057b5-470e-4688-aecc-7925592732b1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:23.352629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:23.353261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:23.353545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:23.353818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:23.354487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:23.355938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae40069-df4f-42d6-96f5-e73d5c9503ea None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:23.382324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 33/61] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:23 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:26.526665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c127b4-6881-44ba-8166-3513f6b1af3a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.527207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c127b4-6881-44ba-8166-3513f6b1af3a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.527207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c127b4-6881-44ba-8166-3513f6b1af3a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.528824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 29/62] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:26.534142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a0b2-a378-4b59-acd1-47f59f6c59ad None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.534554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a0b2-a378-4b59-acd1-47f59f6c59ad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.534703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760a0b2-a378-4b59-acd1-47f59f6c59ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.553784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8760a0b2-a378-4b59-acd1-47f59f6c59ad None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:26.554986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8760a0b2-a378-4b59-acd1-47f59f6c59ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:26.577112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:26.655174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 34/63] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:26.690788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c127b4-6881-44ba-8166-3513f6b1af3a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.729592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529b6b2-826f-4f0c-bf72-44d649c778f1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.729905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529b6b2-826f-4f0c-bf72-44d649c778f1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.730158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529b6b2-826f-4f0c-bf72-44d649c778f1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.731683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 35/64] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:26.739973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b3ac-903b-4e08-a14e-93773068c986 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.740667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b3ac-903b-4e08-a14e-93773068c986 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.741289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b3ac-903b-4e08-a14e-93773068c986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.772837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0df9b3ac-903b-4e08-a14e-93773068c986 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:26.774410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0df9b3ac-903b-4e08-a14e-93773068c986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:26.815938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c127b4-6881-44ba-8166-3513f6b1af3a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:26.855120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:26.856132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.856581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.857106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.870278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:26.874789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 36/65] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:26.884319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:26.885405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:26.886340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6d87b-a22f-4448-b706-4feeccc9e705 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6d87b-a22f-4448-b706-4feeccc9e705 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6d87b-a22f-4448-b706-4feeccc9e705 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 37/66] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:26.886919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:26.887246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:26.888828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 30/67] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:26.898414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:26.911178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:26.913447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:26.931132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.005403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 38/68] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:27.013871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.019906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e814ad-28c7-44e6-8b2b-67554e5fefb5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.031633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.062426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.063209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.063391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.063674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.064529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:27.064943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.066693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9111f-a842-4ee1-9297-538a4afa71e3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.081282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 31/69] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:27.083007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cc117-9cf0-437c-a0bb-e87d8302b0db None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.083419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cc117-9cf0-437c-a0bb-e87d8302b0db None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.083703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cc117-9cf0-437c-a0bb-e87d8302b0db None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.085047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 32/70] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:27.085754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cc117-9cf0-437c-a0bb-e87d8302b0db None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.138186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31810ef7-954f-4151-a1f4-cfda80be1242 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.175216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.175867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.176160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.176443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.178759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.187810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:27.193648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:27.194475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:27.195237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 39/71] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:27.195861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.206811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cc117-9cf0-437c-a0bb-e87d8302b0db None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.240088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.240730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.241050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.241331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.251223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-36e40a89-c9a1-40b5-9497-4ff50a43d53f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:27.256913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:27.257939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:27.258884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 33/72] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:27.259655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.259655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.259789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.266541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.277717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:27.278939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:27.301267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab22ffb-d45d-40ad-8814-5e269851d0b1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.335987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.336569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.336858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.337170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.346252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 34/73] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:27.347170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.352510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.354029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cded1c6-a882-4996-aa82-7b687f7d2343 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.355692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 40/74] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.356915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34ccc-cd67-462e-ba40-cd74fa91ae7d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.357232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34ccc-cd67-462e-ba40-cd74fa91ae7d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.357516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e34ccc-cd67-462e-ba40-cd74fa91ae7d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.358740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 41/75] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:27.359437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e34ccc-cd67-462e-ba40-cd74fa91ae7d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.428031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.457868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d7261-e04e-4f99-af1b-d3e0603f43a6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.464833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e34ccc-cd67-462e-ba40-cd74fa91ae7d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.489806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.490635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.491359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.491665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.492062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.492782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:27.493119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.494506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.498985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.499602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.499886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.500164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.500884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:27.501221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.502938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 35/76] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.503494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abcaebf-d007-4838-bf76-737beb9ffbf0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.503715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.514588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 42/77] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.515780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.516072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.516356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.534627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:27.535840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:27.600460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 43/78] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:27.601251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.610109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5f5e7-9dbc-465a-8ed7-607d2a8bed96 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.644801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.645463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.645752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.646029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.648660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.713340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571f30-7bc4-42cb-997b-6b3620fccdca None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.714827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.716313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.730159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.747430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.748067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.748347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.748620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.758695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:27.763624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:27.764738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 44/79] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:27.765773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.816800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 36/80] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:27.817666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.850447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:27.873746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685780d-aff1-4b5d-bbbc-15878d572dea None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.906086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.906725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.907287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.909205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.910677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.912286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 45/81] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.913075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.918349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cf4f5-1dbb-4963-ae0f-b79f67d294f4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:27.952165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:27.952804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:27.953125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:27.953408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:27.955375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:27.956851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:27.958512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 37/82] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:27.959277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:27.965766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.018443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4848d-5c81-4d92-b101-52484f4f2dfe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.052749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.053426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.053723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.053996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.055146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.056616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.058430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 46/83] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.059007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.059818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5956a7-0c37-4f2a-9ac6-8ec6dde51151 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.061364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.099834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.100466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.100756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.101071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.101798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:28.102109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.103491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.112166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 38/84] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.112876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.161843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260f15d-0244-48e9-8915-6757a57ec2b9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.175797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.196124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.196764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.197074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.197361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.198522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.199977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.201667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 47/85] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.202429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.216241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8889f3-fd9b-4795-9267-9d4a3ad7adac None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.249537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.250158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.250448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.250728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.280836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.299222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5815ed-99c0-4a27-a67c-07fdd81698ae None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.318937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.320452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.333821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.334441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.334723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.334999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.383074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.401361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.402832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.418400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 39/86] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:28.419134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.480044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.507217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 48/87] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:28.508071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.519705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec3fd1-6e25-4b48-bd3d-3e849dac322a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.552690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.553333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.553611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.553890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.554562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.555999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.583811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 40/88] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:28 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:28.584625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.599397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.621723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8caaf-5f90-4d11-9eed-2c215420e0e1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.662131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.662793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.663081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.663352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.664588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.666125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.668002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 49/89] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.668780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.706962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb37c3-f8e4-4b13-98b8-0625e08af5e3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.729843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.749721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.750426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.750625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.750884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.751746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:28.752054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.754271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.759901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 41/90] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=cfc89a51451c403880880749d727c042 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.760770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.796039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599df543-b8b2-4539-9d4c-b732b1aea5cd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.840339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.841165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.841524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.841724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.842994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.844510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.847642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 50/91] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.848429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.874516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:28.925977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacf85d-e576-4974-934b-b7bba62c7447 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:28.971711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:28.972491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:28.972782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:28.973113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:28.974408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:28.976044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:28.979347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 42/92] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:28.980186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:28.990531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac768-9f5b-480b-9a27-8988c56176fa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.035906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.036544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.036830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.037142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.051198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.134181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da226dd7-1b4f-46e5-92ce-43def8641177 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.161917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.163406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.196723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.199311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.199961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.200248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.200536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.201378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=8e99a25baa6a4ee4875a0277c6006aec) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:29.201695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.203170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.214470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 43/93] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/role_assignments?user.id=8e99a25baa6a4ee4875a0277c6006aec&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:29.215293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.333986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6e76-e6b6-4d4b-a9bf-9a97417ab37c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.353866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.371973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 51/94] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:29.373588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.374226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.374545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.374798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.375611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:29.375934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.378076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.384395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 44/95] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:29.395678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.531070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f901d10-8cb8-4bcb-80cb-bd319db8323c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.553429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:29.574672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.575345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.575626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.575906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] PATH_INFO: `/v3/projects/275305b57a4042428becc9ac8f2b12e4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.577174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] RBAC: Authorizing `identity:get_project(project_id=275305b57a4042428becc9ac8f2b12e4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.578640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e661c33-1a0c-4355-8aee-ff4b7a37952b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.580986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 52/96] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/projects/275305b57a4042428becc9ac8f2b12e4 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:29.586819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:29.704141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea54cd-c250-4f1e-92d0-3a34dd0c9a01 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:29.741502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:29.742915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:29.743400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:29.743872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:29.744898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=275305b57a4042428becc9ac8f2b12e4, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:29.745445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:29.747092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb801aa-2202-48ba-a887-87841d185503 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:29.753359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 45/97] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:29 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=275305b57a4042428becc9ac8f2b12e4&role.id=cfc89a51451c403880880749d727c042 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:31.614322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520f10-4f7a-47c5-9dda-3b435466ef5c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.614831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520f10-4f7a-47c5-9dda-3b435466ef5c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.615094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24520f10-4f7a-47c5-9dda-3b435466ef5c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.617008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 53/98] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:31.622398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827695-4603-44a3-ba20-e3861f39c1d8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.622819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827695-4603-44a3-ba20-e3861f39c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.623188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72827695-4603-44a3-ba20-e3861f39c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.650083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-72827695-4603-44a3-ba20-e3861f39c1d8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:31.651407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-72827695-4603-44a3-ba20-e3861f39c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:31.705072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:31.750471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 46/99] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:31.765829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a892-08c0-422d-8860-e0f766a6fbc8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.766122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a892-08c0-422d-8860-e0f766a6fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.766411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a892-08c0-422d-8860-e0f766a6fbc8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.768746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 54/100] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:31.774187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:31.774553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:31.774931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:31.788859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a892-08c0-422d-8860-e0f766a6fbc8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:31.805393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:31.807325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:31.943780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 47/101] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:31.992402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.008811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a892-08c0-422d-8860-e0f766a6fbc8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.093244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.093902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.094184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.094458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.129983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f386fdef-136a-402b-a8fa-78c5fc38a6eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:32.135009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:32.135853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:32.136690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:32.139124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 55/102] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:32.139932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f389-6719-44d0-b4e1-4a7c6bf56062 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.140902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cdd3-8341-4650-a832-06c9444999b9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.141417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cdd3-8341-4650-a832-06c9444999b9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.141882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cdd3-8341-4650-a832-06c9444999b9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.142524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.143918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 56/103] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:32.148786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cdd3-8341-4650-a832-06c9444999b9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.181850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.182552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.182783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.183067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.184278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.185831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc82a9d-8090-4f0e-be1f-8d8ebaaf3099 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.187548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 48/104] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:32.189563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f39d0d-4c17-4052-9707-d2cbf309a0c5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.189563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f39d0d-4c17-4052-9707-d2cbf309a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.189768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f39d0d-4c17-4052-9707-d2cbf309a0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.209031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-96f39d0d-4c17-4052-9707-d2cbf309a0c5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:32.210122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-96f39d0d-4c17-4052-9707-d2cbf309a0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:32.210991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.271341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cdd3-8341-4650-a832-06c9444999b9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.281269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 49/105] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:32.282413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4daea-2ae9-43eb-b5b2-597b71aa0d07 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.282736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4daea-2ae9-43eb-b5b2-597b71aa0d07 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.283104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4daea-2ae9-43eb-b5b2-597b71aa0d07 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.284537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 50/106] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:32.285442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45e0ec-5c1f-4e06-ac2e-4fb44842e39a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.285731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45e0ec-5c1f-4e06-ac2e-4fb44842e39a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.286011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45e0ec-5c1f-4e06-ac2e-4fb44842e39a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.291254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 51/107] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:32.291254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45e0ec-5c1f-4e06-ac2e-4fb44842e39a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.319711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.320442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.320785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.321175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.321961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:32.322352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.323814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09ab8e-a69c-4167-96d8-14304d42ca8c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.332997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 57/108] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:32.338659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967d27-a109-4b9c-a42f-8cb335027dd1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.338947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967d27-a109-4b9c-a42f-8cb335027dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.339354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967d27-a109-4b9c-a42f-8cb335027dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.357505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-28967d27-a109-4b9c-a42f-8cb335027dd1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:32.358681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28967d27-a109-4b9c-a42f-8cb335027dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:32.403665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45e0ec-5c1f-4e06-ac2e-4fb44842e39a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.405237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.439531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 58/109] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:32.441195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a59eee-0795-4744-8195-30383760ddfa None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.441943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a59eee-0795-4744-8195-30383760ddfa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.442415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a59eee-0795-4744-8195-30383760ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.455287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824932-c740-40c1-810e-36a94e37039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.456657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824932-c740-40c1-810e-36a94e37039f None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.457072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824932-c740-40c1-810e-36a94e37039f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.457424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824932-c740-40c1-810e-36a94e37039f None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.458326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df824932-c740-40c1-810e-36a94e37039f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.460318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df824932-c740-40c1-810e-36a94e37039f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.463501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-62a59eee-0795-4744-8195-30383760ddfa None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:32.464798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-62a59eee-0795-4744-8195-30383760ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:32.492354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 52/110] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/projects => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:32.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824932-c740-40c1-810e-36a94e37039f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.521004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.560113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 59/111] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:32.561569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a59eee-0795-4744-8195-30383760ddfa None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.644105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824932-c740-40c1-810e-36a94e37039f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.681582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.682219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a59eee-0795-4744-8195-30383760ddfa None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.682652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.682652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.682876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.696890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:32.702446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:32.703278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:32.704031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 53/112] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:32.704686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.723209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.733607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.734260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.734544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.734822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] PATH_INFO: `/v3/users/4c2aa344758c4d7c8833a3f71b2561f5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.751021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2aa344758c4d7c8833a3f71b2561f5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:32.753508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:32.755329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 60/113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/users/4c2aa344758c4d7c8833a3f71b2561f5 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:32.756199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.785789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:32.836065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb67c5-2386-42b0-bd7e-82d333910d7d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.872494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.873366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.873756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.874134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.879504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37566a7-a3e7-493e-8133-4cbe25eb29f6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:32.886822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:32.892492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:32.893493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:32.894464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 54/114] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:32.895890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.921497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:32.922428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:32.922712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:32.922992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:32.938096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:32.945609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:32.947445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 61/115] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:32.949632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:32.990003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.025895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9f04-0582-495a-a2f4-88e65f149316 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.047688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.063541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.064353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.064531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d51c45-dbfa-433d-93e7-4d25cf527b4e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.064747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.065061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.065788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.066098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.067751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.084388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 55/116] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.084388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.112617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.113828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.114119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.114503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.187258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.189397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3f90e-cdf3-4176-83e2-64522bfa9f61 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.193802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9dd2-2099-4f13-afc7-b32d3d0986c7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.230349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.230967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.231253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.231534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.232256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.232565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.233996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.239921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.245187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 56/117] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.246694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.283567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.306507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 62/118] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:33.307613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69399b8-8ae0-4dbd-9453-a142c8291f7b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.307903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69399b8-8ae0-4dbd-9453-a142c8291f7b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.308183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69399b8-8ae0-4dbd-9453-a142c8291f7b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.309687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 63/119] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:33.313859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69399b8-8ae0-4dbd-9453-a142c8291f7b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.367867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50607-411d-4940-8744-441495e5e0e3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.404866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.405519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.405798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.406072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.406782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:33.407086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.409034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.417144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 57/120] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.417931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.430130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69399b8-8ae0-4dbd-9453-a142c8291f7b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.471141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.471814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.472142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.472422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.473618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.475115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.476699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 64/121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.477522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.486472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.530091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc781a-4c32-47eb-a312-fc4e18ec615b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.538161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.568541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.569458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.569913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.570477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.571984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.573760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be43e04-220b-4008-bb02-353d4b71b192 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.575716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 58/122] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.577024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.577312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.577589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.589340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a6b0-e854-4b4d-8e82-5e881a7154c2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.597213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:33.598424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:33.624029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.624947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.625425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.625869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.627214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.628828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.630702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 65/123] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.631625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.684985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 59/124] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:33.684985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.749718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e36c1a-9fa8-433f-8eee-5b7806c66047 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.750288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.775628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:33.788174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.788765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.789076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.789359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.790542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.792007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.793817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 66/125] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.794421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.805670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84b801-ac7f-4887-97b0-a426d13c6a79 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.843185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.844137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.844671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.845312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.846578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.848016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.854404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 60/126] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.854404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.902119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383bdb4-3899-4463-9566-d33c7d6c272e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.937854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:33.938493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:33.938772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:33.939576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:33.940734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:33.942483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:33.948702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 67/127] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:33.948702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:33.961361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac50a2-9eac-4c60-baa7-95f04ba8fef1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:33.989155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.000216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.000834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.001160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.001989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.003157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=4c2aa344758c4d7c8833a3f71b2561f5) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:34.003475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.004925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.006797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.012180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 61/128] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/role_assignments?user.id=4c2aa344758c4d7c8833a3f71b2561f5&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=5ee6a512846a4cb289a858610ac49509 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:34.013187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.069840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d7b2f-8efc-4007-bb1c-61586cb6292d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.125361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.126139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.126440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.126728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.138438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93c4d5-a503-4eba-96bf-c222d51013d6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.142748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:34.148794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:34.149733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:34.150486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 68/129] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:34.151191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.182319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.183459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.183955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.184441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.185952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.187653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.190396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 62/130] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:34.196596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.269910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.295098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8643365-6a9e-4cdc-b747-7b005a76267d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.321667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7540e8-8010-4d46-bdb5-e27b2cb13bed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.322016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.347188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.347877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.348164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.348446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/657440e9843c4f8097ef973d0719da30/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.351625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, user_id=657440e9843c4f8097ef973d0719da30, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.353148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.366012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.367120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.367725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.368235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/8e99a25baa6a4ee4875a0277c6006aec/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.369930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 69/131] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:34 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/657440e9843c4f8097ef973d0719da30/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:34.370882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, user_id=8e99a25baa6a4ee4875a0277c6006aec, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.370988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.372856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.391425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 63/132] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:34 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/8e99a25baa6a4ee4875a0277c6006aec/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:34.397691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.502605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5432d7-d4f8-46da-9c5b-a6ecd87a5e7a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.558011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.558668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.558955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.559236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.559985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:34.560299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.561767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.584616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 70/133] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:34.588499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665116b-81df-4dfd-bdb8-537e2001d2e0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.590841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.611483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.635469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.636188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.636480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.636768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/657440e9843c4f8097ef973d0719da30/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.639397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, user_id=657440e9843c4f8097ef973d0719da30, role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.641362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.659579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 64/134] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:34 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/657440e9843c4f8097ef973d0719da30/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:34.685959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:34.768936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1e2c-0c8f-4b08-9ae2-1305da339e8a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:34.817409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:34.818074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.818364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.818649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.819892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:34.821398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d66d94-0c93-468c-9fb9-6f3094261ad8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:34.823201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 71/135] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:34.828902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:34.871691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc39c4-36a3-4a28-8da3-d5efd8481972 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.871987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc39c4-36a3-4a28-8da3-d5efd8481972 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.872179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc39c4-36a3-4a28-8da3-d5efd8481972 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.873598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 72/136] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:34.886318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:34.886640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:34.886888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:34.917299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:34.918520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:35.016074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80032f3b-ab80-4cf7-a60c-6a86dcacbdda None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.054554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 73/137] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:35.077100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.077777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.078064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.078348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] PATH_INFO: `/v3/projects/275305b57a4042428becc9ac8f2b12e4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.080222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] RBAC: Authorizing `identity:get_project(project_id=275305b57a4042428becc9ac8f2b12e4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.081872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.083561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 65/138] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:34 2022] GET /identity/v3/projects/275305b57a4042428becc9ac8f2b12e4 => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:35.093349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.104862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.184308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.201752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.339155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb693408-cd66-4727-bb43-42968ce65d4b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.342001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e05e3-c28e-4c0c-87a9-c8ced69649dc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.423830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.424668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.425072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.425358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.426529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.427982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.429666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.430365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.430654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.430937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] PATH_INFO: `/v3/projects/275305b57a4042428becc9ac8f2b12e4/users/657440e9843c4f8097ef973d0719da30/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.434559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=275305b57a4042428becc9ac8f2b12e4, user_id=657440e9843c4f8097ef973d0719da30, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.435968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238fcf6-af10-4acb-b21a-9fc54fa52fa6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.437732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 66/139] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:35.444310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:35.460817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 74/140] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:35 2022] PUT /identity/v3/projects/275305b57a4042428becc9ac8f2b12e4/users/657440e9843c4f8097ef973d0719da30/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:35.599411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a7766-93ea-4274-91e3-821cdd4c988e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:35.622137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:35.654764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:35.655424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:35.655709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:35.655986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:35.656705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:35.658295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3423bb9-e83d-466b-99b9-b6f1f2f32e65 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:35.696612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 67/141] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:35 2022] POST /identity/v3/users => generated 283 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:37.007833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca888f-cc29-4300-9767-35e4459cd173 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.008367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca888f-cc29-4300-9767-35e4459cd173 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.008450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca888f-cc29-4300-9767-35e4459cd173 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.009973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 75/142] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:37.015770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928bb-6f8d-46c6-94f0-813eb760774c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.016232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928bb-6f8d-46c6-94f0-813eb760774c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.016330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d928bb-6f8d-46c6-94f0-813eb760774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.038928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-89d928bb-6f8d-46c6-94f0-813eb760774c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:37.040148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-89d928bb-6f8d-46c6-94f0-813eb760774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:37.137259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 68/143] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:37.173634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca888f-cc29-4300-9767-35e4459cd173 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.292271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.297885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca888f-cc29-4300-9767-35e4459cd173 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.335644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.336306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.336589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.336869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.364589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:37.367357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec07a-8690-4613-9008-990502e346bb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.367876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec07a-8690-4613-9008-990502e346bb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.368312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaec07a-8690-4613-9008-990502e346bb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.369792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 69/144] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ac765f3b-328d-434f-9326-35e6a1442f55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:37.374791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:37.375688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:37.377415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 76/145] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:37.382361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaec07a-8690-4613-9008-990502e346bb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.384503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd611523-882f-4b45-b309-590c9a2bfd03 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.384789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd611523-882f-4b45-b309-590c9a2bfd03 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.385098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd611523-882f-4b45-b309-590c9a2bfd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.410542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cd611523-882f-4b45-b309-590c9a2bfd03 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:37.411877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd611523-882f-4b45-b309-590c9a2bfd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:37.508332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 77/146] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:37.509899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017d4a-a7cd-41ab-a484-f72a9c7161a3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.510186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017d4a-a7cd-41ab-a484-f72a9c7161a3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.510463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017d4a-a7cd-41ab-a484-f72a9c7161a3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.511989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 78/147] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:37.518019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.518457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.518740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.540010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:37.541274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:37.590668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.596405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaec07a-8690-4613-9008-990502e346bb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.620328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 79/148] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:37.621147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.659805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:37.678090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.679902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.680255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.680620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.682211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:37.682536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:37.684677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d307d-dae2-42b4-b272-facfa285cc6c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:37.709135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 70/149] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:37.710635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86df6-c6ed-4255-ad7a-1babbb9d8a5d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.710919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86df6-c6ed-4255-ad7a-1babbb9d8a5d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.711192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e86df6-c6ed-4255-ad7a-1babbb9d8a5d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.712416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 71/150] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:37.719419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e86df6-c6ed-4255-ad7a-1babbb9d8a5d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:37.855874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf249b1-fc9e-49e0-911f-cf888ebd649f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.944621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:37.945302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.945637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.945851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:37.950848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e86df6-c6ed-4255-ad7a-1babbb9d8a5d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:37.976663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-73a2e39a-0718-4bae-bcec-5e21e2c7470b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:37.981707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:37.982540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:37.983292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 80/151] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:37.984685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:37.984685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:37.984685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.028921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:38.030281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:38.053138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.053789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.054078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.054355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.086670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:38.103437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:38.104330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:38.105096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 72/152] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:38.105729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.169010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.231166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 81/153] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:38.232280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.265076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.292916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281d8f-e798-4c51-8518-25d3093b4d67 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.366706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.367370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.367654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.367930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] PATH_INFO: `/v3/users/8e99a25baa6a4ee4875a0277c6006aec` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.369148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] RBAC: Authorizing `identity:get_user(user_id=8e99a25baa6a4ee4875a0277c6006aec)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.371377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.372757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 73/154] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/users/8e99a25baa6a4ee4875a0277c6006aec => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.378031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.418053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388fdce-b779-4a14-8fb8-a52db409a9e5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.478730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.479588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.480014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.480446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.481341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:38.481790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.483345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.501330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 82/155] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.502987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.561308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.573823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071598a-0aac-481b-8ab1-b31276f2f941 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.612993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.653853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5a26d-41ad-4510-9d85-18f3248aec8a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.664873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.665630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.665917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.666193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.666926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:38.667234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.668739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.684684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 74/156] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.685587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.699574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.700516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.701018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.701488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.726255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:38.731562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:38.732410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:38.741310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:38.743217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 83/157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:38.744234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.840782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9138d5-dd55-437a-8e42-732cd5f7e0f7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.889769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.890419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.890704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.891082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.899573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7115-be02-45db-becc-ad5f2d47b901 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:38.921084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.947818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:38.960864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:38.961632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:38.961921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:38.962201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:38.963437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.965010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:38.966703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 84/158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:38.967491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:38.974820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:38.976314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.084441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 75/159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:39.085339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.144373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.195893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b9d22-f551-41e2-beeb-b3c616248f8c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.196822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb810cb-68fc-472d-8ecd-e59d91ae7a9e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.238162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.238803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.239083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.239359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.240091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:39.240397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.241830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.260273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 76/160] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.261915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.278240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.278895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.279176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.279454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.280634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.282197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.284250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 85/161] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:38 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.288561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.374307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.386636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de71ce-2fcb-4f4d-a81f-44f8eb0d2b07 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.402104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.413255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0113c-e8a0-4a67-92ad-2779ca0ec32b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.445759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.446630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.447074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.447505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.463112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.463775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.464060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.464341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.469708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.471935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac55091-6cbb-4501-88f2-8df6e10600ac None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.473863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 86/162] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.474848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cce93-13f6-4b58-8a8b-d5dc3111cce6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.475139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cce93-13f6-4b58-8a8b-d5dc3111cce6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.475411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cce93-13f6-4b58-8a8b-d5dc3111cce6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.476789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 87/163] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:39.477661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cce93-13f6-4b58-8a8b-d5dc3111cce6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.525460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.527290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.636679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 77/164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:39.637571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.654033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.690175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cce93-13f6-4b58-8a8b-d5dc3111cce6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.729659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.730583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.731119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.731536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] PATH_INFO: `/v3/users/4c2aa344758c4d7c8833a3f71b2561f5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.733178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2aa344758c4d7c8833a3f71b2561f5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.734977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.736948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 88/165] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/users/4c2aa344758c4d7c8833a3f71b2561f5 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.737967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:39.741166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.754891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be3d52-81e7-4c96-b0c9-f8254a66db82 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.793583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.794519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.794951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.795364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.863541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9575ef-c665-49b6-9efd-9883cfce6373 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:39.887343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.889157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.900154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:39.901051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.901500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.901925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.902863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=8e99a25baa6a4ee4875a0277c6006aec) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:39.903316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:39.904931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3024d-4a38-4519-8bca-0f321bc1cdc4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:39.905564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:39.914415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 89/166] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/role_assignments?user.id=8e99a25baa6a4ee4875a0277c6006aec&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:39.915918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d637-e1ed-450a-ba51-0af70f0f520a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:39.916347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d637-e1ed-450a-ba51-0af70f0f520a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:39.916762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914d637-e1ed-450a-ba51-0af70f0f520a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:39.936286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e914d637-e1ed-450a-ba51-0af70f0f520a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:39.937783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e914d637-e1ed-450a-ba51-0af70f0f520a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:40.005934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 78/167] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:40.006651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.009350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.021800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 90/168] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:40.023540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8c2a-2c1f-48f9-8f52-30f72e1ed0cc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.024043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8c2a-2c1f-48f9-8f52-30f72e1ed0cc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.024512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b8c2a-2c1f-48f9-8f52-30f72e1ed0cc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.026145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 91/169] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:40.027113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8c2a-2c1f-48f9-8f52-30f72e1ed0cc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.145466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b8c2a-2c1f-48f9-8f52-30f72e1ed0cc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.151123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09a1f-5fdb-43e5-8853-fc6561f8bf90 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.167980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.182127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.182769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.183300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.183716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.185057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.186665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe275e-abef-49ce-ab57-f46b723d1de2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.189598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 92/170] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.191506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0297266-41d8-485d-b5bf-851915edfa76 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.192224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0297266-41d8-485d-b5bf-851915edfa76 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.192670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0297266-41d8-485d-b5bf-851915edfa76 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.192941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.193741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.194026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.194303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.194889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 93/171] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:40.195748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0297266-41d8-485d-b5bf-851915edfa76 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.198117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.198958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10fed9-0226-4422-871b-2cd059aaaa56 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.200823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 79/172] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.202145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.202430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.202705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.223031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:40.224236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:40.269461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.295722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 80/173] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:40.296342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.324137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0297266-41d8-485d-b5bf-851915edfa76 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.357090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.357727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.358006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.358284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.359437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.361765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.363514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 94/174] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.364250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.401497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a3716-d738-4f4d-b3eb-5f5813685424 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.415003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.436296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.436940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.437251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.437525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.448490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-89b0ff6a-b821-46ea-b51b-4ac7de2dc1ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:40.453522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:40.455066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:40.455984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 81/175] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:40.456623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5940d2-80ff-4413-b52f-66584d16eeda None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.457268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5940d2-80ff-4413-b52f-66584d16eeda None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.457268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5940d2-80ff-4413-b52f-66584d16eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.468522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ce6e2-6d30-407d-849e-b37c0b4d9c08 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.474598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ea5940d2-80ff-4413-b52f-66584d16eeda None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:40.475796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ea5940d2-80ff-4413-b52f-66584d16eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:40.498850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.505351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.506018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.506298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.506575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/4c2aa344758c4d7c8833a3f71b2561f5/roles/5ee6a512846a4cb289a858610ac49509` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.508540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, user_id=4c2aa344758c4d7c8833a3f71b2561f5, role_id=5ee6a512846a4cb289a858610ac49509)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.510045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.534679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 95/176] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:40 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/users/4c2aa344758c4d7c8833a3f71b2561f5/roles/5ee6a512846a4cb289a858610ac49509 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:40.534679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.550342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 82/177] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:40.551480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891c32-5537-4582-83e0-21aa5f3d3fc4 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.551768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891c32-5537-4582-83e0-21aa5f3d3fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.552042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891c32-5537-4582-83e0-21aa5f3d3fc4 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.553540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 83/178] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:40.554135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891c32-5537-4582-83e0-21aa5f3d3fc4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.667245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87486426-5b25-4eac-8f93-17406abad023 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.694646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11891c32-5537-4582-83e0-21aa5f3d3fc4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.696356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.712427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.713419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.713939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.714311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.715122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.715435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.716888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.727590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 96/179] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=cfc89a51451c403880880749d727c042 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.727590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.738313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.739189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.739568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.739936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.753428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:40.759265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:40.760230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:40.761278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 84/180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:40.762747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.810775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:40.874771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d3336-5d32-4f5a-ac97-9f5a21b9af81 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.896805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637b3fa-c7ee-4eac-b0f9-39fa5d9c105b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:40.927086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.928775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.929120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.929404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.930490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.930809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.932572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6df282-c9d8-4e66-85a1-56b2d785218f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.942892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 97/181] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.943064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:40.943767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.944045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:40.944251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.944363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:40.944618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.944721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:40.945727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:40.946126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:40.947863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:40.958856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 85/182] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:40.959873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:40.975457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:40.977135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:40.994696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.085031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 98/183] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:41.086204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.087962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f82d68-e880-49af-affa-3ac4825940fc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.147516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.148173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.148451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.148731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.160154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:41.166014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:41.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:41.167662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 86/184] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:41.168316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.174835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.238734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1eb667-e5b6-4c5a-af36-e2bf82710559 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.270266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.280707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.282083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.283179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.283619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.284486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:41.284934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.286787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.288807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09131b67-596e-4dfb-a6b8-66c454849bd7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.296691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 99/185] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.297777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.333334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.334351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.334837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.335509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] PATH_INFO: `/v3/users/657440e9843c4f8097ef973d0719da30` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.337038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] RBAC: Authorizing `identity:get_user(user_id=657440e9843c4f8097ef973d0719da30)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.338702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.340641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 87/186] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/users/657440e9843c4f8097ef973d0719da30 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.341623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.404005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6ba4-ca56-43cb-9212-27363c509d52 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.418745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.443118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.443767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.444051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.444328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.445936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.447778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.450883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 100/187] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.450883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.462371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f95a909-af79-4653-a181-572f1d26e819 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.498475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.499134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.499461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.499864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.500934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:41.502630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.504581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.513749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 88/188] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.514547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.530889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.560191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f1645-5990-4784-8de4-a309e20e0e10 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.595383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.596150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.596433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.596717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.609660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.611684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.613743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 101/189] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:41.614573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.629743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e510d-301c-4771-9750-c46cbb11e1e5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.659698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.667703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.668350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.668633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.668957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.725530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804d3c-e273-4dc7-ac6f-dd1e5f56e631 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:41.747832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.749415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.768306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.772858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:41.773532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:41.773804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:41.774083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:41.774800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:41.776670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:41.809028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 102/190] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:41 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:41.809028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.880290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 89/191] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:41.881505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:41.943067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:41.953643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcbead-c431-4dbf-b25c-0700511325bf None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.004359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.004935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.005244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.005522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.006837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.006980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7d689-a61c-4b10-a567-761a02039669 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.008369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.010599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 103/192] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.011057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.026421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.055015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.055661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.055946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.056225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.137573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319d0a-8952-4036-93a7-ced4359346a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.139504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.141282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.176419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.177107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.177396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.177672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] PATH_INFO: `/v3/projects/275305b57a4042428becc9ac8f2b12e4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.178966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] RBAC: Authorizing `identity:get_project(project_id=275305b57a4042428becc9ac8f2b12e4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.180540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.182621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 104/193] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/projects/275305b57a4042428becc9ac8f2b12e4 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.183572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.220673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.263625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 90/194] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:42.265177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.281978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.319737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cb38c-758d-45bf-b1f8-5637381a4c62 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.364342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.364996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.365729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.366017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.391760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b8cd3-cc89-4f55-812a-6ecb5c3d4a1a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.428702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.429720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.430201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.430671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.431660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=275305b57a4042428becc9ac8f2b12e4, user.id=657440e9843c4f8097ef973d0719da30) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:42.432199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.433865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.440814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 91/195] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/role_assignments?user.id=657440e9843c4f8097ef973d0719da30&scope.project.id=275305b57a4042428becc9ac8f2b12e4&role.id=cfc89a51451c403880880749d727c042 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.442293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.462830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.464681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.498444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.559417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.584315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 105/196] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:42.589542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.590035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8746-217f-4117-9bf4-14f476dbf75a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.636997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.637709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.637987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.638260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.639431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.641569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.644849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 92/197] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.651111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.712293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50621730-a834-4008-adbb-cdc704de44c6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.747338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.756142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.757164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.757669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.758142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.759959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.761758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.764059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 106/198] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:42.770043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:42.787874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea1747-98b0-49f3-ace7-60e77efcdee6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.834795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.835338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.835614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.835887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.836684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=01dcf5e723e846f69d1d1b628ac4e296) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:42.837008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.839043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256a2cc-f36b-45ba-9c2e-38e9f69ecfc4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.847500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 93/199] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/role_assignments?user.id=01dcf5e723e846f69d1d1b628ac4e296&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:42.848876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766030-ceef-480c-8890-ce50140b81cd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.849185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766030-ceef-480c-8890-ce50140b81cd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.849459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f766030-ceef-480c-8890-ce50140b81cd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.851498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 94/200] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:42.857443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.857632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.857831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.885488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:42.887083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:42.893064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:42.906095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fddf6-06f2-4211-84be-9b7555687349 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:42.957702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:42.958340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:42.964505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:42.964834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:42.965781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=01dcf5e723e846f69d1d1b628ac4e296) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:42.966090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:42.967663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:42.980722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 107/201] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:42 2022] GET /identity/v3/role_assignments?user.id=01dcf5e723e846f69d1d1b628ac4e296&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:43.007874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 95/202] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:43.031172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.112756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.161333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7ceaf-b74b-40d2-bff9-53a3cd523607 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.199632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.200301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.200590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.200870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.213442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:43.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:43.219825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:43.220699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 108/203] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:43.227836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.336872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93e5f0-ce27-4584-828d-ea9312203ebb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.378386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.379381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.379868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.380333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.381271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:43.381790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.383343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.397235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 96/204] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:43.403133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.545728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3103ee-a317-4c01-a531-360f81cbff97 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.568990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.583214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.583878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.584163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.584443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] PATH_INFO: `/v3/users/4c2aa344758c4d7c8833a3f71b2561f5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.589711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] RBAC: Authorizing `identity:get_user(user_id=4c2aa344758c4d7c8833a3f71b2561f5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.594332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.596759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 109/205] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/users/4c2aa344758c4d7c8833a3f71b2561f5 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:43.602565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:43.607288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.726517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19081e67-423e-49b5-b755-77e54cbf9cc4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:43.767592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:43.768576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:43.769076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:43.769542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:43.864430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:43.865931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:43.867939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:43.998115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 97/206] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:44.003543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:44.144194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9953ca-b773-4d64-a164-82d47b04522a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:44.189273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:44.189928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.190213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.190490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.191931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:44.193426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89842b0f-a9f8-4983-8b40-6cff6cac4a46 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:44.195396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 110/207] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:44 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:44.201201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:44.314434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bb229-04c5-4138-bd6d-a9a543a36758 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:44.315836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:44.351650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:44.352259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.352536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.352816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.353653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d, user.id=4c2aa344758c4d7c8833a3f71b2561f5) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:44.353963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:44.355409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b342725-38ea-4af5-b4f9-daee99291816 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:44.362288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 98/208] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:44 2022] GET /identity/v3/role_assignments?user.id=4c2aa344758c4d7c8833a3f71b2561f5&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=5ee6a512846a4cb289a858610ac49509 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:44.753740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e371-fb5a-4ac4-836d-733798afa587 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.754275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e371-fb5a-4ac4-836d-733798afa587 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.754275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e371-fb5a-4ac4-836d-733798afa587 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.757519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 111/209] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:44.762841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.763139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.763424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.790507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:44.791763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:44.864471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fedc1-2700-45cb-8296-02051cb6e859 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.864789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fedc1-2700-45cb-8296-02051cb6e859 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.865068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fedc1-2700-45cb-8296-02051cb6e859 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.866571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 112/210] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:44.869818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 99/211] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:44.871736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:44.872026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:44.872302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:44.891392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:44.892739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:44.896118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:44.960650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:44.976063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 113/212] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:45.000386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.015842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.025392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de19c1-1108-4039-afa8-292ebd5cc97b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.076618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.077545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.078258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.078640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.096623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:45.105917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:45.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:45.107596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:45.108188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 100/213] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:45.115253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.128903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215d7d-ce8d-4e74-8c1f-8cff1d608809 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.166137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56196725-749f-460d-a881-e07891718409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.167131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196725-749f-460d-a881-e07891718409 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.167636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196725-749f-460d-a881-e07891718409 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.168101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196725-749f-460d-a881-e07891718409 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.169590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56196725-749f-460d-a881-e07891718409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.171238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56196725-749f-460d-a881-e07891718409 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.177206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 114/214] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:45.179698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56196725-749f-460d-a881-e07891718409 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.238521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074580db-f139-4d5d-b248-ed63dff3b121 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.257218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.275313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.276288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.276783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.277281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.278201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:45.278699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.280301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.290107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 101/215] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:45.295770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.313151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56196725-749f-460d-a881-e07891718409 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.317116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.364473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.365145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.365431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.365710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.366855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.368608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d71d7-ff71-44cc-b0c0-0428592ec631 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.401458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 115/216] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:45 2022] POST /identity/v3/users => generated 297 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:45.419290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033e1f8-180c-41a9-9399-7ba81589b8ad None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.426679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1541-4b57-45d9-9939-276ebaf2dde0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.426974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1541-4b57-45d9-9939-276ebaf2dde0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.427262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e1541-4b57-45d9-9939-276ebaf2dde0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.428631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 116/217] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:45.434557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.434845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.435346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.457103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:45.458331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:45.462104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.463019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.463503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.463953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] PATH_INFO: `/v3/users/c1a1bf25c5524d5da8dbb3c59736202f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.481726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] RBAC: Authorizing `identity:get_user(user_id=c1a1bf25c5524d5da8dbb3c59736202f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.483568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.486100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 102/218] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/users/c1a1bf25c5524d5da8dbb3c59736202f => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:45.491960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.537921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 117/219] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:45.562284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.578046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.605901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7791-62a5-4ecc-8a09-6ac22dfb4d79 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.612685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.648389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.649063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.649323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.649598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.683224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03caa-d67b-4fef-8f84-8f5fbe23c9db None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.730657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.731303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.731574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.732483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.736902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.738803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96645e4-0745-44cd-a0ae-418c8601346a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.743500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b0c89fac-140b-45ef-9905-9166b3e29fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:45.748447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:45.749358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:45.750167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 118/220] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:45.752084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44350-6d7c-4f29-8af7-e6e3d081f784 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.752084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44350-6d7c-4f29-8af7-e6e3d081f784 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.752084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44350-6d7c-4f29-8af7-e6e3d081f784 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.754000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 119/221] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:45.758027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44350-6d7c-4f29-8af7-e6e3d081f784 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.836850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.865435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 103/222] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:45.867161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.867673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.868172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.878627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44350-6d7c-4f29-8af7-e6e3d081f784 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:45.880292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:45.889176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:45.890743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:45.914809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:45.915409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:45.915690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:45.915966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:45.916819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:45.917221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:45.918775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:45.928588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 120/223] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:45.929397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:45.968501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 104/224] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:45.969692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.057484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67221b4-118e-46a5-9579-c66661bb66b5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.076611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6dc22-8193-43f0-aeca-738f02718c59 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.085374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.109167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.110353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.111202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.111693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.113821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.115487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.117500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 121/225] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.118534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.123150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.123406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.123633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.123960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.125367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.126921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.128748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 105/226] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:45 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.129636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.182583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12a1be-7f61-44aa-a291-48a83d089e91 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.253252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f49b-6cc8-4046-9abc-e53b61b1a4f7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.284316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.285228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.285485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.285838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.296810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:46.299434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.300281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.300748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.301233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.302212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=c1a1bf25c5524d5da8dbb3c59736202f) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:46.302699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:46.303343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:46.304195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:46.305062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 122/227] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:46.305726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.307409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.316300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 106/228] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/role_assignments?user.id=c1a1bf25c5524d5da8dbb3c59736202f&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=cfc89a51451c403880880749d727c042 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.317732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.368766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.495198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6877d1-fa42-4b89-9af5-a633ca43eaa5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.510110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.518222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2fcd0-a3db-4a0d-abd3-0b7abca6178a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.545773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.546411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.546875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.547276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.548598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:46.548924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.550857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.569666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.570317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.570598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.570876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.571419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 107/229] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.572066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.573646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.575158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.577027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 123/230] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.587692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.765902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:46.773156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc3f0-f3ee-4b09-8b45-b5d06eaeb5be None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.808981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e9f06-e523-4e95-8997-ef262e2b8fe9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:46.815487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.816149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.816430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.816955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.818395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.819673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.826390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 108/231] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:46.827095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:46.882215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:46.882934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:46.883217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:46.883490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/01dcf5e723e846f69d1d1b628ac4e296/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:46.886836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, user_id=01dcf5e723e846f69d1d1b628ac4e296, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:46.888319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:46.915640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 124/232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:46 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/01dcf5e723e846f69d1d1b628ac4e296/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:46.965957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2fde-e5ac-4ed9-ad4d-d73baa8873c9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.008552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.009241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.009524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.009799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.010991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.012473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0504ac5-3de4-493c-99e4-c54a8451dfd6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.014284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 109/233] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:46 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:47.020824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:47.052916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:47.160315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ecd51-d538-4f5a-8906-a42b0b32955d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:47.199247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:47.200272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.200754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.201271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/01dcf5e723e846f69d1d1b628ac4e296/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.203700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, user_id=01dcf5e723e846f69d1d1b628ac4e296, role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:47.205527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99062920-6d3d-4dad-81fa-831255e365f0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:47.221987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 125/234] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:47 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/01dcf5e723e846f69d1d1b628ac4e296/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:47.817254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328229-431c-497e-a395-f148bcf5f794 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.818281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328229-431c-497e-a395-f148bcf5f794 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.818765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3328229-431c-497e-a395-f148bcf5f794 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.820543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 110/235] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:47.826039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b994d-fc37-4d84-9f27-455f6ce6b38b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:47.826821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b994d-fc37-4d84-9f27-455f6ce6b38b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:47.827302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b994d-fc37-4d84-9f27-455f6ce6b38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:47.857093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b82b994d-fc37-4d84-9f27-455f6ce6b38b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:47.858599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b82b994d-fc37-4d84-9f27-455f6ce6b38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:47.953769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 126/236] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:47.997634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328229-431c-497e-a395-f148bcf5f794 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.040698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a0d08-f6d4-4c42-81c4-bfd8e8b2a3db None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.041365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a0d08-f6d4-4c42-81c4-bfd8e8b2a3db None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.041839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a0d08-f6d4-4c42-81c4-bfd8e8b2a3db None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.043399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 127/237] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:48.049860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.062116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.062400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.062676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.100887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:48.102452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:48.118815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.166570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3328229-431c-497e-a395-f148bcf5f794 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.202701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.203354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.203636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.203910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.222271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 128/238] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:48.225251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:48.230721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:48.231612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:48.232381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:48.236902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 111/239] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:48.238282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.254184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.381270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.387051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431489e0-5772-4b87-b359-8a4aade68394 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.415653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bf32e-6279-4eca-8181-5e00eeb67e64 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.457689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.458351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.458631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.458909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.460250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.462426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.465567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 112/240] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.471606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.492980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.493615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.493897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.494168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.494899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:48.495203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.496851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.513673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 129/241] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.520524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.523886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.655304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30b413-f05f-42de-a986-e9a545c40454 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.687327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9dda0-51c2-4a4b-9ff8-33a484e7faa4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:48.718225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.718860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.719137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.719411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.720249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.722285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f1833-0a8c-446d-b277-06876d38fbb3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.746356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 113/242] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:48 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:48.758656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:48.759625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.759962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.760250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] PATH_INFO: `/v3/users/b207072f2fb1411e9541ea3cf223782e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.786448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c31cb-534e-4018-95f5-0dfe1f3ab5ca None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.786761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c31cb-534e-4018-95f5-0dfe1f3ab5ca None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.787104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c31cb-534e-4018-95f5-0dfe1f3ab5ca None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.788933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 114/243] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:48.793200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] RBAC: Authorizing `identity:get_user(user_id=b207072f2fb1411e9541ea3cf223782e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:48.794834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:48.796188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:48.796473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:48.796806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:48.807385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 130/244] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/users/b207072f2fb1411e9541ea3cf223782e => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:48.812042 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.834386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:48.836255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:48.854170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.917709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 115/245] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:48.942553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:48.951422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:48.995154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737dca-8750-48dc-99e4-270e8ded1489 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.034840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.035470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.035749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.036018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.072042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052da1a-f292-4d6f-a483-544fd697302a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.116155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.117242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.118208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.118322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.118784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.119130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.137331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:49.144404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:49.145500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:49.146437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:49.148578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 116/246] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:49.154247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.170184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.201046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.259757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 131/247] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:49.265660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.272279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42aaf8-383c-4d2f-82f4-f57109afc1e8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.311542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.312163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.313322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.313322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.314383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:49.314829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.316223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.326319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 117/248] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.332854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.394668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d3284-1c14-40ca-b0b9-03c8d632d22c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.417597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.437716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.439282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.440298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.440879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.442998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.445047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.447507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 132/249] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.452686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f4417-7acb-4482-8638-bb7a7daa22ca None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.457958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.494010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.503502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.505428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.505889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.506343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] PATH_INFO: `/v3/users/c1a1bf25c5524d5da8dbb3c59736202f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.507730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] RBAC: Authorizing `identity:get_user(user_id=c1a1bf25c5524d5da8dbb3c59736202f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.509455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ad407-2793-410d-be48-6940372ff54c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.511927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 118/250] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/users/c1a1bf25c5524d5da8dbb3c59736202f => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.512857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece904-18a3-4751-aed3-5e03c4c97638 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.513712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece904-18a3-4751-aed3-5e03c4c97638 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.514195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece904-18a3-4751-aed3-5e03c4c97638 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.515738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 119/251] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:49.518924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece904-18a3-4751-aed3-5e03c4c97638 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.598402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729dbca-e2d8-4405-b044-539ac266d7be None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.641151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece904-18a3-4751-aed3-5e03c4c97638 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.646884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.647759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.648237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.648734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.650299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=b207072f2fb1411e9541ea3cf223782e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:49.650817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.653277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff5829-0ad2-4e21-adc2-ec8738cdeb6c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.669001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 133/252] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/role_assignments?user.id=b207072f2fb1411e9541ea3cf223782e&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=5ee6a512846a4cb289a858610ac49509 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.669001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f66c-8ce3-492a-a054-b3747ac90798 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.669001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f66c-8ce3-492a-a054-b3747ac90798 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.669001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f66c-8ce3-492a-a054-b3747ac90798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.703049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-84b8f66c-8ce3-492a-a054-b3747ac90798 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:49.704873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-84b8f66c-8ce3-492a-a054-b3747ac90798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:49.705601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.706485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.707332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.707817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.714020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.717204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.721586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 120/253] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:49.729509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:49.753435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.887353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96eb517-24f1-404e-aae3-0afde42b4896 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:49.894463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 134/254] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:49.895592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1286-77ee-437c-a18d-2d7ce4a7299f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.895876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1286-77ee-437c-a18d-2d7ce4a7299f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.896147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a1286-77ee-437c-a18d-2d7ce4a7299f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.897711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 135/255] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:49.914394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.914668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.914937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.935193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:49.936370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:49.944735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:49.945466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:49.945749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:49.946060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/c1a1bf25c5524d5da8dbb3c59736202f/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:49.949002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, user_id=c1a1bf25c5524d5da8dbb3c59736202f, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:49.950911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:49.958669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:49.976945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 121/256] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:49 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/c1a1bf25c5524d5da8dbb3c59736202f/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:54:49.977938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.077526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 136/257] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:50.129376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.146112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.208895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79dbb1-2473-4157-a81c-d806e938cb8f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.247946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.248949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.249476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.249953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.261363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:50.266220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d99751-eb1a-40c8-a5dc-dae11f731ec1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:50.267817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:50.268696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:54:50.269513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:50.272058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 122/258] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:50.283712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.314920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.315633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.315919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.316192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.333073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:50.342257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:50.343162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:50.343934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 137/259] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:50.351697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.374352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.404500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bf4b-612e-4f46-bdf0-4cd4849a9c5f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.433083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.443500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.444669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.446293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.447429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.448351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:50.449498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.451297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.467652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 123/260] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.476363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.529237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95309a-b0d3-4b44-8035-51518f1c0c81 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.585942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00968541-37c1-4880-9190-0e18c128f724 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.598630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.599547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.600019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.600501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.601445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:50.601959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.603546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.617493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 138/261] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:50.623056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.668684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.672311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.672791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.673833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.675223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.680988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.683865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 124/262] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.691372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:50.808281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.832850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:50.881011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c490cd7-a9ee-443f-aefd-195ecc40434d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.893140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c522f6-ad85-45f1-88ae-593e7ef0ea73 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:50.944989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.944989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.944989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.944989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.951351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:50.952012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:50.952300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:50.952583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] PATH_INFO: `/v3/users/01dcf5e723e846f69d1d1b628ac4e296` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:50.956848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] RBAC: Authorizing `identity:get_user(user_id=01dcf5e723e846f69d1d1b628ac4e296)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:50.958363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:50.960012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 139/263] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/users/01dcf5e723e846f69d1d1b628ac4e296 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:50.971964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.083844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.085345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.188312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16354af1-1dc2-4f58-8dfa-9c8787128259 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.223486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.244213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.244837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.245156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.245432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.262721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.300886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 125/264] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:51.321602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.330996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.332414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.439430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 140/265] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:51.445264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.493331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.557572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c33eb-e114-4027-853d-39b79d25505b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.596296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.596930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.597241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.597515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.598704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.598886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070f3ac-16c3-414b-b6f8-5e8acc86c361 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.600160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.602810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 141/266] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:51.609379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.683927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.684589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.684879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.685280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.689771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.691249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.693676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 126/267] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:51.715636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:51.751392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481ecd-9551-4bca-a931-4ad329556c5c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:51.764117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.776875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:51.794977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:51.796045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.796547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.797112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.799124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=01dcf5e723e846f69d1d1b628ac4e296) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:51.800312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:51.803564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b785c-ea34-4ecd-8e7c-2bc9788fd511 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:51.812343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 142/268] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/role_assignments?user.id=01dcf5e723e846f69d1d1b628ac4e296&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:51.882441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87918-42ad-4cad-8254-3e61f409f72a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.882742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87918-42ad-4cad-8254-3e61f409f72a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.883251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e87918-42ad-4cad-8254-3e61f409f72a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.889371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 143/269] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:51.890879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:51.891165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:51.892229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:51.916632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:51.918474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:51.937164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe333c-5ec9-4628-88bf-7d79ca593b4f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.009436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 144/270] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:52.040745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.044673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.046489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.046489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.046489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.046489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=01dcf5e723e846f69d1d1b628ac4e296) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:52.046845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.048779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.056244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 127/271] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:51 2022] GET /identity/v3/role_assignments?user.id=01dcf5e723e846f69d1d1b628ac4e296&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.177363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62385670-8e93-4a8e-afc2-b6ef2c1f1b2c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.181011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.218740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.220410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.221181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.221801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.234034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:52.240147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:52.241055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:52.241850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:52.244468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 145/272] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:52.250246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.357949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.368827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27adaf-623e-455f-8464-76d85e834192 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.404746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.405410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.405692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.405970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.406703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:52.407012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.408434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174f528-12e2-45bb-a5a2-89c495d7ec31 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.421077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 128/273] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.426135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.554488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08eefc2-486b-49f0-a021-7bbc610bbd7f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.563489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6fdf-1e7a-4e0d-aeed-a0786b5f528e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.563796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6fdf-1e7a-4e0d-aeed-a0786b5f528e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.564057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c6fdf-1e7a-4e0d-aeed-a0786b5f528e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.567226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 129/274] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:52.571377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.571664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.571945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.591746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:52.592382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.593081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:52.593620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.594096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.594568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] PATH_INFO: `/v3/users/274b6eba61a449d38cfe37dc19a98026` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.609509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] RBAC: Authorizing `identity:get_user(user_id=274b6eba61a449d38cfe37dc19a98026)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.611834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.614170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 146/275] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/users/274b6eba61a449d38cfe37dc19a98026 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.620498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.681395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.688989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 130/276] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:52.707710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.745365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d385-3217-451f-8670-e65b9d5cd240 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.781772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.783279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.783617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.783946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.784872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, user.id=274b6eba61a449d38cfe37dc19a98026, scope.system=all) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:52.785502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.787822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87cd1f-8b9b-4300-97f7-ac1d4262a53f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.800995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 147/277] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/role_assignments?user.id=274b6eba61a449d38cfe37dc19a98026&scope.system=all&role.id=cfc89a51451c403880880749d727c042 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:52.810963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:52.822225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a133-810f-4424-be21-9cb4b5d2256c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.822561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a133-810f-4424-be21-9cb4b5d2256c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.822900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0a133-810f-4424-be21-9cb4b5d2256c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.824528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 148/278] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:52.832195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602331d7-a30a-4fe8-a519-68de09780fe9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.832483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602331d7-a30a-4fe8-a519-68de09780fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.832759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602331d7-a30a-4fe8-a519-68de09780fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.837903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a1fea-5150-4111-9f10-d7d2c01069fc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:52.856998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-602331d7-a30a-4fe8-a519-68de09780fe9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:52.858593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-602331d7-a30a-4fe8-a519-68de09780fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:52.886434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:52.887398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.887681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.887960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.890004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:52.892465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:52.895748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 131/279] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:52.903094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:52.945499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 149/280] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:52.946959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5d66-17c3-4fc8-b54f-c74b109ecdd3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.947253 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5d66-17c3-4fc8-b54f-c74b109ecdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.947534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab5d66-17c3-4fc8-b54f-c74b109ecdd3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.953248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 150/281] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:52.958978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:52.959734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:52.960146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:52.981012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:52.982178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:52.996127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.027257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf293e5-f527-4406-85a8-db886fc42056 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.056785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 151/282] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:53.057814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.072871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.073774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.074230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.074692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.075551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.078173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.105709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 132/283] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:54:52 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:53.106826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.117813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.210739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0965d1f-d6e9-457c-aecf-2c5fc366fe23 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.237203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ba2b0-ace0-4495-8d10-15fbc0f91ee5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.263219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.264203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.264492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.264769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.278547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:53.278892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.279575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.279860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.280138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:53.284127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:53.285052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:53.285989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:53.288029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 152/284] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:53.294375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.300008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:53.306567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:53.306716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:53.307624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:53.308656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 133/285] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:53.315346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.405159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.436080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e2c03-cd50-4a32-90fc-567650f65349 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.451814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb9a9a-7c52-4337-9b6c-f19bcf047de1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.484068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.484687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.485012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.485695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.486049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:53.486350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.488316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.496916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 153/286] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.503162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.504061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.504800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.505178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.505528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.506334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:53.506711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.508134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.519990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 134/287] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:53.525714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.574708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.612685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613cfd-61bc-468e-b79e-7e84b7706655 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.648722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.657301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed0cab-61b8-4ac1-b7cf-29147c28d811 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.658216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.658583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.658939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] PATH_INFO: `/v3/users/c1a1bf25c5524d5da8dbb3c59736202f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.660303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] RBAC: Authorizing `identity:get_user(user_id=c1a1bf25c5524d5da8dbb3c59736202f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.662013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.663716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 154/288] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/users/c1a1bf25c5524d5da8dbb3c59736202f => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.669137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.688042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.698443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.699353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.699809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.700288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] PATH_INFO: `/v3/users/b207072f2fb1411e9541ea3cf223782e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.701708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] RBAC: Authorizing `identity:get_user(user_id=b207072f2fb1411e9541ea3cf223782e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.703348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.705581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 135/289] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/users/b207072f2fb1411e9541ea3cf223782e => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.711202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:53.828789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217b1f-d8de-4984-bab8-9b0f4f1a6f6b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.893544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69559140-bcbb-4b3f-8c3c-1781e334cc62 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:53.896407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.897059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.897338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.897613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.903757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:53.950292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:53.950859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:53.951136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:53.951404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:53.953554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:53.955728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:53.965049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 136/290] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:53.966985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.036312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.038465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.098564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.188534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 155/291] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:54:54.193979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.197529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98b2f4-40c2-4d75-87a3-0f6d8c9a67ff None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.258295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.269780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.270488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.270774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.271055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/b207072f2fb1411e9541ea3cf223782e/roles/5ee6a512846a4cb289a858610ac49509` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.274284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, user_id=b207072f2fb1411e9541ea3cf223782e, role_id=5ee6a512846a4cb289a858610ac49509)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.276763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.299739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 137/292] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:54:53 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/users/b207072f2fb1411e9541ea3cf223782e/roles/5ee6a512846a4cb289a858610ac49509 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:54:54.366710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecfba7-7df7-4c99-8c5b-8183b242557a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.410060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.411079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.411587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.412064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.413543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.415324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15aa14-87a3-424f-a9b8-9981431fba24 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.417685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 156/293] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:54.426119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:54.546767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44522d-6d82-4b63-a4d1-36dfad6bc89b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:54.574393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:54.589180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:54.590005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:54.590479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:54.591002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:54.591810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=c1a1bf25c5524d5da8dbb3c59736202f) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:54.592126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:54.593628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24929444-ece1-441c-8f51-0f9ef7179d34 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:54.600652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 138/294] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:54 2022] GET /identity/v3/role_assignments?user.id=c1a1bf25c5524d5da8dbb3c59736202f&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=cfc89a51451c403880880749d727c042 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.022349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c7-920a-477d-a07e-e859badd475b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.022859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c7-920a-477d-a07e-e859badd475b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.023273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbb3c7-920a-477d-a07e-e859badd475b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.025051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 157/295] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:55.030044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc881-921f-4030-9759-db050e45b057 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.030335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc881-921f-4030-9759-db050e45b057 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.030615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bc881-921f-4030-9759-db050e45b057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.055377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d38bc881-921f-4030-9759-db050e45b057 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:55.056566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d38bc881-921f-4030-9759-db050e45b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:55.155488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 139/296] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:55.188086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbb3c7-920a-477d-a07e-e859badd475b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.308257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbb3c7-920a-477d-a07e-e859badd475b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.310640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.347397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.348082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.348358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.348624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.358972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:55.364155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:55.365062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:55.365854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 158/297] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:55.374041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc881-921f-4030-9759-db050e45b057 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.476068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bc881-921f-4030-9759-db050e45b057 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.493805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.511650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.512274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.512558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.512831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.513584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:55.513890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.515263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49c7bd4-d7ba-4c84-8558-6e60c6eba8dd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.524650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 140/298] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.530205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.611557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4e39-10da-41e5-9ae1-9bedee6526c0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.611734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4e39-10da-41e5-9ae1-9bedee6526c0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.612150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4e39-10da-41e5-9ae1-9bedee6526c0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.614084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 141/299] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:55.618923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.619206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.619484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.637729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:55.638951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:55.640169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b5da9-090b-4da5-b922-f5b66e70b355 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.684085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.685139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.685704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.685986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] PATH_INFO: `/v3/users/274b6eba61a449d38cfe37dc19a98026` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.687379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] RBAC: Authorizing `identity:get_user(user_id=274b6eba61a449d38cfe37dc19a98026)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.689765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.697097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 159/300] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/users/274b6eba61a449d38cfe37dc19a98026 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:55.698721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.711263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 142/301] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:55.735099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.736997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.825374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3dea67-eb42-4467-aadd-d3afe68d2c5b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.849217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07ccec-f947-4a79-a138-4b18f23c69ec None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:55.860633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:55.865515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.866378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.866658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.866933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] PATH_INFO: `/v3/system/users/274b6eba61a449d38cfe37dc19a98026/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.869007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=274b6eba61a449d38cfe37dc19a98026, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.870397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba1d6-8d59-4474-97e0-469897bd8644 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.885054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 160/302] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Jul 27 19:54:55 2022] PUT /identity/v3/system/users/274b6eba61a449d38cfe37dc19a98026/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:54:55.886933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d55ec-7416-4d4d-9741-15d699d70cb5 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.887298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d55ec-7416-4d4d-9741-15d699d70cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.887650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d55ec-7416-4d4d-9741-15d699d70cb5 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.889315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 161/303] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:55.895870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.896252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.896445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.898173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:55.899172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:55.899673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:55.900206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:55.901901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:55.903586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:55.905593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 143/304] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:55.912101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:55.922234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:55.923768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:56.009000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 162/305] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:56.028004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.043486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f6f90-ecd3-406d-9d4f-5cfb01233b1a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.061067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.098501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.099490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.100494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.101032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] PATH_INFO: `/v3/groups` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.102177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.104633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.131262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 144/306] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Jul 27 19:54:55 2022] POST /identity/v3/groups => generated 229 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:54:56.175072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a015fb-c24c-45e1-acf3-ac59c8f7b5ae None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.215250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.215918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.216202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.216482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.228996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:56.234338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:56.235231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:56.236047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 163/307] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:56.241930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.287656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.360253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887df930-9a24-4e2c-9056-f534332a59d9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.396491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.397165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.397448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.397718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.398439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:56.398737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.400123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.409918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 145/308] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:56.415178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.502939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.518880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c241c8-cc91-4c3c-a4ab-1980ef6247bb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.553271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.553909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.554185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.554457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] PATH_INFO: `/v3/users/b998cce424ce4089a5a7a49a6871b953` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.569024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] RBAC: Authorizing `identity:get_user(user_id=b998cce424ce4089a5a7a49a6871b953)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.570485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dddfb3-67dd-42dd-b5e8-32bb2c23be37 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.572520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 164/309] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/users/b998cce424ce4089a5a7a49a6871b953 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:56.581220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.668558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b332db-2309-46d7-a5af-8dbc76da311c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.668741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b332db-2309-46d7-a5af-8dbc76da311c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.669048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b332db-2309-46d7-a5af-8dbc76da311c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.670507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 165/310] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:56.678782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.678782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.678782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.690358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:56.693769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff86a-f16d-48fe-a8f3-2fa11c78e4fa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.699186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:56.700377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:56.732678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.733697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.734177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.734661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.736248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, user.id=b998cce424ce4089a5a7a49a6871b953, scope.system=all) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:56.736834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:56.738504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:56.744316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 146/311] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/role_assignments?user.id=b998cce424ce4089a5a7a49a6871b953&scope.system=all&role.id=5ee6a512846a4cb289a858610ac49509 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:56.780783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 166/312] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:54:56.807345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:56.928841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd6e6e-de3f-4c83-bbb5-0c6a2c0d3e4a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:56.965819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:56.966703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:56.967137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:56.967557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:56.978585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:56.984255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:56.985357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:56.986349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:56.988629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 147/313] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:56.994537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.081640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.107381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a1a05-ec13-479a-a55e-f2c3e95263af None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.144610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.145255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.145707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.145983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.146700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:57.146999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.148370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.157211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 167/314] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.162741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.191283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.286728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1923c9-60c7-41d9-ad16-f98687a8012c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.323769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.324721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.325267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.325768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] PATH_INFO: `/v3/users/b207072f2fb1411e9541ea3cf223782e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.327153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] RBAC: Authorizing `identity:get_user(user_id=b207072f2fb1411e9541ea3cf223782e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.328805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.330968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 148/315] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/users/b207072f2fb1411e9541ea3cf223782e => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.336630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.439424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef2fc-78cd-4f9e-9738-35d39e5a8ca8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.474518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.475076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.475356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.475632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.479681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.545468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.546935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.648134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 168/316] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:54:57.654047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.751939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.761941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd217ae6-a0dd-40a6-94e5-878154f703e4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.797952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.799067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.799605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.800104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.801531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.803182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924c127-17f5-4792-88c4-4cc39a7b11d2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.805443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 149/317] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:57.811438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:57.847168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:57.915359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ac5b5-9751-4d4a-b86e-ee4a16cd65b1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:57.951486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:57.952105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:57.952381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:57.952652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:57.953461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, scope.project.id=e8dd370bebd048bd88eca9183841c3e3, user.id=b207072f2fb1411e9541ea3cf223782e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:57.953767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:57.956225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5fd1a-75f9-404e-aec6-ddb755c9cb93 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:57.962554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 169/318] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:54:57 2022] GET /identity/v3/role_assignments?user.id=b207072f2fb1411e9541ea3cf223782e&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=5ee6a512846a4cb289a858610ac49509 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:58.078680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4cf-ca60-411b-80f1-6c284b6e1108 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.079801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4cf-ca60-411b-80f1-6c284b6e1108 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.080299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc4cf-ca60-411b-80f1-6c284b6e1108 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.082263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 150/319] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:58.088126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.088766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.089284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.109735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:58.111585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:58.181974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 170/320] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:58.207088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc4cf-ca60-411b-80f1-6c284b6e1108 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.314230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfc4cf-ca60-411b-80f1-6c284b6e1108 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.349313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.349884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.350165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.350441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.361156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-17f81896-0f10-4501-b466-81f69f547ae9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:58.366154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:58.367048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:54:58.368253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 151/321] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:54:58.373419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.403448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.420230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eabe0e-5e69-432e-8925-7a5351181622 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.420513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eabe0e-5e69-432e-8925-7a5351181622 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.420794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eabe0e-5e69-432e-8925-7a5351181622 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.422199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 152/322] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:54:58.430235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.430518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.430794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.450357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:58.451548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:58.467212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.484353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392acc3-5ab4-4eec-83b0-eb932f255beb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.522219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.523070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.523640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.524083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.524941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:58.525416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.525518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 153/323] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:58.527013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.535788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 171/324] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:58.541638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.547221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.646613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545eb7f-1eb4-42b4-b48d-1926112ddd1e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.648392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.659007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc799-808b-4d65-9b5d-25a81952caee None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.682842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.683489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.683771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.684055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] PATH_INFO: `/v3/users/274b6eba61a449d38cfe37dc19a98026` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.685267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] RBAC: Authorizing `identity:get_user(user_id=274b6eba61a449d38cfe37dc19a98026)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.686729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.688354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 154/325] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/users/274b6eba61a449d38cfe37dc19a98026 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:58.694525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.700438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.701117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.701402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.701675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.702820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.704835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.707093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 172/326] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:58.712442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:58.719002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:58.813837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04febe6-17bd-445a-b7f7-9384b2e6a170 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.814029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc501a-d4ac-4e9c-b7b9-64bc4f4d94cd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:58.849215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.849962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.850311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.850603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] PATH_INFO: `/v3/groups` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.851265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.852497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:58.852719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e8d04-12ee-455a-a072-1a3e5790aac6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.853138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.853417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.853688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.854469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] RBAC: Adding query filter params (role.id=cfc89a51451c403880880749d727c042, user.id=274b6eba61a449d38cfe37dc19a98026, scope.system=all) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:58.854775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:58.856184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a9829-f613-454a-8503-9e159c54b030 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:58.877717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 173/327] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Jul 27 19:54:58 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:54:58.877925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 155/328] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Jul 27 19:54:58 2022] GET /identity/v3/role_assignments?user.id=274b6eba61a449d38cfe37dc19a98026&scope.system=all&role.id=cfc89a51451c403880880749d727c042 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:54:58.878004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd29dc2-3400-4207-9c1c-4835ba6c8bb4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.878004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd29dc2-3400-4207-9c1c-4835ba6c8bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.878004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd29dc2-3400-4207-9c1c-4835ba6c8bb4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.878004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 174/329] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:54:58.879281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:58.879577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:58.879853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:58.903610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:54:58.904784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:54:58.988758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 156/330] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:54:59.011075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd29dc2-3400-4207-9c1c-4835ba6c8bb4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:59.035023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:59.129731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd29dc2-3400-4207-9c1c-4835ba6c8bb4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:59.165456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:59.166029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:59.166313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:59.166591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:59.179968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:54:59.185522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:54:59.186398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:54:59.187322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 175/331] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:54:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:54:59.193092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:59.201813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:59.305094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7f12-0c16-4d81-8e12-1af983602b1f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:59.342291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:59.343026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:59.343216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:59.343470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:59.344199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:54:59.344504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:59.345962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:59.354944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 157/332] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:54:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:59.361157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:59.453628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:59.475706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe549ef-2c72-41c2-81a5-6fac01865281 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:59.511751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:59.512389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:59.512674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:59.512950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] PATH_INFO: `/v3/users/b998cce424ce4089a5a7a49a6871b953` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:59.514170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] RBAC: Authorizing `identity:get_user(user_id=b998cce424ce4089a5a7a49a6871b953)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:59.515642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd7289-135f-495f-a0bd-1800599416de None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:59.517738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 176/333] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:54:59 2022] GET /identity/v3/users/b998cce424ce4089a5a7a49a6871b953 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:54:59.523385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:54:59.600534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:54:59.636440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb35c4e-7316-4df1-bad1-5d302401b725 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:54:59.673415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:54:59.674079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:54:59.674356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:54:59.674630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] PATH_INFO: `/v3/system/users/b998cce424ce4089a5a7a49a6871b953/roles/5ee6a512846a4cb289a858610ac49509` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:54:59.676094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b998cce424ce4089a5a7a49a6871b953, role_id=5ee6a512846a4cb289a858610ac49509)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:54:59.677532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5387b-047b-4f76-820c-cb76d6b2b2b4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:54:59.688432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 158/334] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Jul 27 19:54:59 2022] PUT /identity/v3/system/users/b998cce424ce4089a5a7a49a6871b953/roles/5ee6a512846a4cb289a858610ac49509 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:55:01.647542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbff5-d218-4887-b3fc-d77c2854f616 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.648568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbff5-d218-4887-b3fc-d77c2854f616 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.649130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbff5-d218-4887-b3fc-d77c2854f616 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.651004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 177/335] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:01.656074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.656915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.657974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.686419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:01.687991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:01.766286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 159/336] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:01.807696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cbff5-d218-4887-b3fc-d77c2854f616 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:01.931864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cbff5-d218-4887-b3fc-d77c2854f616 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:01.979503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:01.980529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:01.981063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:01.981554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:01.994570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2392f251-70ef-4c2b-b3bf-a58c9a55f41a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:02.000350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:02.001958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:02.002940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:02.005659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 178/337] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:02.011136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.113665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.138055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a76a2-efcc-4e7f-bbf7-7c6d9ffee1e8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.138702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a76a2-efcc-4e7f-bbf7-7c6d9ffee1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.139177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a76a2-efcc-4e7f-bbf7-7c6d9ffee1e8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.140838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 179/338] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.153540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e858e3-d550-4861-bb0c-8de0186e9206 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.153540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e858e3-d550-4861-bb0c-8de0186e9206 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.153540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e858e3-d550-4861-bb0c-8de0186e9206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.236205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b6e858e3-d550-4861-bb0c-8de0186e9206 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:02.249155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b6e858e3-d550-4861-bb0c-8de0186e9206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:02.258007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.267059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a03b72-da8d-4b13-a99f-d8b556786b5e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.414558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.415123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.415411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.416180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.416534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:02.416847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.418272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc05ae-9c17-44fc-b350-f08df8dcab44 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.444193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 160/339] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:02.448205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996d2de-b7a2-4807-9ae3-e23d629907ab None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.448205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996d2de-b7a2-4807-9ae3-e23d629907ab None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.448205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996d2de-b7a2-4807-9ae3-e23d629907ab None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.456994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 161/340] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.456994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f815-7fae-4d8f-8494-c01b3c12c9eb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.456994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f815-7fae-4d8f-8494-c01b3c12c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.456994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f815-7fae-4d8f-8494-c01b3c12c9eb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.462238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 162/341] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.463521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f815-7fae-4d8f-8494-c01b3c12c9eb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.464994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 180/342] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:02.465839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7e65-6c20-42bf-b6c7-9d97c5623e33 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.468352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7e65-6c20-42bf-b6c7-9d97c5623e33 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.471864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7e65-6c20-42bf-b6c7-9d97c5623e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.504640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c7bd7e65-6c20-42bf-b6c7-9d97c5623e33 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:02.507823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c7bd7e65-6c20-42bf-b6c7-9d97c5623e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:02.669871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f815-7fae-4d8f-8494-c01b3c12c9eb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.682571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 181/343] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:02.683757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.684124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.684485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.698339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.715543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:02.716459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.716910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.717684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.730332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:02.731641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:02.828561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:02.836177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 182/344] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:02.837562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:02.837995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:02.839055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9cde3-2524-407d-a89e-a4e15f52a161 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:02.965053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0596cf0-9d74-4467-96b3-b442fad3fc73 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:02.972187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 163/345] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:02.975327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbeb0c-a950-44d0-bb63-6911aae8013a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:02.975327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbeb0c-a950-44d0-bb63-6911aae8013a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:02.975327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbeb0c-a950-44d0-bb63-6911aae8013a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:02.976198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 164/346] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:02.976530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbeb0c-a950-44d0-bb63-6911aae8013a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.008909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.015969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.016653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.016936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.017242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.027768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:03.034560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:03.035442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:03.036218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 183/347] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:03.036850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.120320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbeb0c-a950-44d0-bb63-6911aae8013a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.129327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.169945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.170599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.170877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.171147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.183049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6237006-ace6-4a7d-bb63-cd20eac1c524 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.183216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:03.188222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:03.189829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:03.190688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 165/348] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:03.191347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.228691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.229377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.229659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.229934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.245399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-965e3d72-302c-4b3f-8d9b-8eda407a341f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:03.253479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:03.254564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:03.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 184/349] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:03.256342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.256342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.256457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.276569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:03.277812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:03.308074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.329899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf4e7-1884-48d2-90fa-0bb5071a9dfc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.358569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 185/350] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:03.359774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.375553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.376673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.377198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.377743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.379592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.381333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.383312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 166/351] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:03.384093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.422477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.484817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079a815-ff68-4f07-b9de-f7fddc3b929e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.518428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d88a7b-c490-4c36-b1e3-2068c31872fd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.534878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.535504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.535784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.536058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.536772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:03.538122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.539428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.548929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 186/352] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:03.550716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.557187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.557925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.558299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.558666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.559498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:03.559896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.561385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.572250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 167/353] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:03.573302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.592095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.671089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179168d-0f93-4587-b98e-c68b7cc0c5f3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.683265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.695045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8511b-6b4a-46be-927c-914bb768b037 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.714319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.715321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.715321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.715497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.716198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:03.716506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.717935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.727004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 187/354] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:03.727653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.736509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.737220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.737505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.737791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.744037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:03.745554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:03.747216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 168/355] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:03.747989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.842993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:03.852773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad160243-2372-4aea-978b-04bcdf651bd3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.870171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ae06-195d-43ec-9e07-d5f44ae2b06c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:03.896372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.897056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.897334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.897607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.910129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:03.910770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:03.911053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:03.911329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:03.913473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:03.920180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:03.921520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 188/356] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:03.922399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:03.962977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.001042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.002088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4490a-ac8e-4751-9495-47733c3ebe0f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.050105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26cb47-f924-4981-bb25-2fc6187d403b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.093930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.095155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.095439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.095712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.102064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.130225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 169/357] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:04.131485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7395d-c260-4e8d-9010-5d48ee2abd5c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.131832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7395d-c260-4e8d-9010-5d48ee2abd5c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.132109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7395d-c260-4e8d-9010-5d48ee2abd5c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.137584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 170/358] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:04.137584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7395d-c260-4e8d-9010-5d48ee2abd5c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.178113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.179545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.221121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.266392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7395d-c260-4e8d-9010-5d48ee2abd5c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.305388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 189/359] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:04.306384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.310718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.311357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.311636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.311911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.368637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.396209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.397741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.428526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd7d3c-20ac-40fb-ba7a-d6139cb059ab None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.457199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.462773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.463397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.463680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.463963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.464753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=cfc89a51451c403880880749d727c042, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:04.465100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.466549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.472405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 190/360] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=cfc89a51451c403880880749d727c042 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.473214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.510022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 171/361] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:04.510721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.597556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee874-84a9-43dc-8a3b-bbfedfec5b0a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.598559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.623928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4520a-1e5c-410c-8f1a-0852234715a9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.633848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.634474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.634756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.635032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.635762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:04.636103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.637528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7448cb-2296-4e51-af41-3e1857899ad2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.650927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 191/362] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.650927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.650927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.650927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.667416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:04.668252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.668643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:04.669474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.669949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.670432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.673487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.675131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.677572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 172/363] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.678628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.729242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.746008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 192/364] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:04.746895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.803639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d876c-d6f7-461d-8f80-10361b4ffdfb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.844268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.845588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.846026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.846330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.847479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.848907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.854991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 173/365] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.854991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.863939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86531567-e352-47e4-acd8-a2ef696c0813 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.868303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.903680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.904324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.904609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.904889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.906176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.908558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:04.914905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 193/366] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:04.915683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:04.955708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc498716-f33a-446c-9c65-7426186866b0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:04.980278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:04.992063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:04.992708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:04.993085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:04.993363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] PATH_INFO: `/v3/users/b998cce424ce4089a5a7a49a6871b953` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:04.994531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] RBAC: Authorizing `identity:get_user(user_id=b998cce424ce4089a5a7a49a6871b953)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:04.995982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.002549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 174/367] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/users/b998cce424ce4089a5a7a49a6871b953 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.002549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.029814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a2520-21cc-4e0c-8323-e92d7359546c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.076495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.077226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.078133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.078133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.079065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.080508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.086961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 194/368] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.086961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.097248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.118873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c38a2-8c37-47b9-8713-90c4cc974971 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.155159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.155862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.156145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.156422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.168681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:05.174806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:05.175902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:05.176730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 175/369] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:05.177424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.187163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4257e8-b2c1-425d-8b84-032ac4ddb97c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.212999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.224773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.225445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.225732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.226005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.227157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.228583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.235169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 195/370] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.235169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.306448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60893e6-16b9-44fe-a913-b57e182e49e3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.334609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac9cb6-d23f-4dbc-9618-87275c06359a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.346443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.347290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.347739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.348282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.350459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.352372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.354158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 176/371] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.355197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.369819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.374895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.375508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.376319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.376600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.377417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] RBAC: Adding query filter params (role.id=5ee6a512846a4cb289a858610ac49509, user.id=b998cce424ce4089a5a7a49a6871b953, scope.system=all) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.377719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.379146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.391462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 196/372] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/role_assignments?user.id=b998cce424ce4089a5a7a49a6871b953&scope.system=all&role.id=5ee6a512846a4cb289a858610ac49509 => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.392948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.461118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.492096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb39c9-795b-4e0d-91bb-29fc57d8a2c7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.523142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643f067-5de9-4191-9116-06b1cddaed1f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.540190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.541230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.541591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.541946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.542837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.543471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.545506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.557119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 177/373] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.557119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.575809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.576172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.576390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.576668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.577448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.577753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.579733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.590827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 197/374] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.591750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.699295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.750127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf2e40-942e-4fb7-9c11-14562e8af207 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.750631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf31c7-6044-4715-801d-f24589da1b26 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.771616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:05.800717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.801459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.801744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.802020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.802824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=cfc89a51451c403880880749d727c042, scope.project.id=e8dd370bebd048bd88eca9183841c3e3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.803136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.803327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.804159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.804564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.805195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.805698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:05.806697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=e8dd370bebd048bd88eca9183841c3e3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:05.807219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:05.808876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:05.810919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 178/375] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=cfc89a51451c403880880749d727c042 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.812215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:05.818848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 198/376] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:05.944606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed673-b84e-4c2c-ab76-fbcff52719fc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:05.991411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:05.992069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:05.992349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:05.992624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.007310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.094180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.097388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.206697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 179/377] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:06.212442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.365055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abefb4a-0203-4677-aa8f-2add8e2dd372 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.430670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.431401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.431684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.431962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] PATH_INFO: `/v3/projects/9e55cde5d6044829b509ee6bccad61b8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.433289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] RBAC: Authorizing `identity:get_project(project_id=9e55cde5d6044829b509ee6bccad61b8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.434756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.436764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 199/378] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/projects/9e55cde5d6044829b509ee6bccad61b8 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.443161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.506260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.585705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589003b-d080-42be-b9ff-fee2d9e855cb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.623651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.624623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.625133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.625625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] PATH_INFO: `/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.633996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] RBAC: Authorizing `identity:get_group(group_id=331f387c2edf4fc7b8b6780d12b1eba3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.635733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94641f2-a3fa-4f94-ab5b-faf5dd681815 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.637615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 180/379] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.643919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.694538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37c8d5-6c7e-474a-bedd-4e89f278a1a7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.695178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37c8d5-6c7e-474a-bedd-4e89f278a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.695646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37c8d5-6c7e-474a-bedd-4e89f278a1a7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.697513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 181/380] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:06.706317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.706904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.707369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.723224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:06.726004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:06.727590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:06.825479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f284-0b8a-41a0-bd2b-bba360420974 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:06.844570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 182/381] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:06.868478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:06.897166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:06.897806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:06.898083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:06.898359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:06.899154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] RBAC: Adding query filter params (group.id=331f387c2edf4fc7b8b6780d12b1eba3, role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e55cde5d6044829b509ee6bccad61b8) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:06.899466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:06.900921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:06.908627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 200/382] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/role_assignments?group.id=331f387c2edf4fc7b8b6780d12b1eba3&scope.project.id=9e55cde5d6044829b509ee6bccad61b8&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:06.946869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.049323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589fc7-8339-411f-a13f-fad4d7b07824 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.093048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.093710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.093992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.094265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.107449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:07.112476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:07.113466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:07.114724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 183/383] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:07.119791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.223506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7abab-d2c0-4e21-928a-6dc01f6ed75d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.232330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.258896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.259514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.259792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.260066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.260781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:07.261130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.262501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.271187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 201/384] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.276724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.374901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:07.384902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7942b-a250-493d-ba6a-7df58de28077 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.423329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.423978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.424262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.424531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.425862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.427304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.429214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 184/385] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.434732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.539566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a61eba-273f-44c3-adf3-de9b98aed9c9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.574111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.574754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.575036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.575312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.576483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.577954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca947239-f8dc-4b03-a794-7090aecaa6da None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.580171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 202/386] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:07 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:07.585938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:07.715038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21793202-0fe0-4d1b-b19f-6847723dd2bd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:07.754370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:07.755049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:07.755330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:07.755606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:07.757584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, group_id=5f9eedc79d5a4d338bf65c9e0f687ff4, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:07.759022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc4bd8-718d-416e-b5f5-e9e0d3d2190d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:07.781087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 185/387] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Jul 27 19:55:07 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:08.221657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315525-45df-4e3e-b822-bc3ec0b993b8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.222187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315525-45df-4e3e-b822-bc3ec0b993b8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.222187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315525-45df-4e3e-b822-bc3ec0b993b8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.223566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 203/388] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:08.228754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacde43-de69-435d-8c33-f1ddd76cd897 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.229173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacde43-de69-435d-8c33-f1ddd76cd897 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.229344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacde43-de69-435d-8c33-f1ddd76cd897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.248044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1aacde43-de69-435d-8c33-f1ddd76cd897 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:08.249293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1aacde43-de69-435d-8c33-f1ddd76cd897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:08.264009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.349346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 186/389] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:08.382212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89315525-45df-4e3e-b822-bc3ec0b993b8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.429403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.435826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cb1f-4cdd-4ae0-b916-cc21df846bad None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.436113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cb1f-4cdd-4ae0-b916-cc21df846bad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.436390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1cb1f-4cdd-4ae0-b916-cc21df846bad None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.438032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 187/390] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:08.444906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.445221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.445543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.468280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:08.470052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:08.557412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 188/391] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:08.557718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89315525-45df-4e3e-b822-bc3ec0b993b8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.579029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.620645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.621242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.621531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.621805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.647945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:08.653824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.654727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:08.655506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:08.658421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 204/392] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:08.664193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.684321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.707553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339eb1b4-5e3b-4035-8629-835a02e5b8bc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.745015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.745971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.746403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.746828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.758705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:08.763724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6ee7743d-eb6a-4959-83b0-aafc81ecdb1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:08.765123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:08.766208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:08.767850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:08.770268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 189/393] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:08.771715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd136ff-d956-4c4b-8e45-29118921b424 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.772181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd136ff-d956-4c4b-8e45-29118921b424 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.772630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd136ff-d956-4c4b-8e45-29118921b424 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.774325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 190/394] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:08.775934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd136ff-d956-4c4b-8e45-29118921b424 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.777147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302d81-7831-40d2-8d28-55c2d0aa7a4a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.817420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.818043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.818326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.818604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.819326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.819636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.821056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b532c-a0a9-4ca5-a734-763288e0caed None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.830513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 205/395] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.835055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.835055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.835055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.857431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:08.858955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:08.898165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd136ff-d956-4c4b-8e45-29118921b424 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:08.934621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:08.935577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:08.936082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:08.936555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:08.937496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:08.938025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:08.940134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:08.949728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 191/396] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:08.950740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:08.955626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:08.969111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 206/397] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:08.970407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.082282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de3299-4a7d-44b9-9a61-925f487f424b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.104775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.117669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef355ac-ae68-430c-8a7a-64739c39ce6e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.125533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.126616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.127264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.127742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.129209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.130962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.137279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 192/398] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.138327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.159422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.159995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.160675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.161000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.162238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.163746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.169032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 207/399] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:08 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.170057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.225718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.262995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49bfe1-5f86-4330-bfe8-7967ef13f603 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.301303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.302095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.302513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.302888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.312187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293025d4-6f45-4ed4-82a0-07a6716c44ae None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.313963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:09.320105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:09.321073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:09.321854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 193/400] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:09.322502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.359965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.361432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.361988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.362323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.363538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.366021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.367723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 208/401] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.368492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.403035 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.444895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9d4af-369c-410e-bf9c-70d30d83dbd0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.480595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.489067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.490275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.490657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.491181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.492676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.494556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.502182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 194/402] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.503152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.504528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b751d5-80dd-416d-b2f6-f3426beb0063 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.544118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.544727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.545028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.545306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.546127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:09.546428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.548319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.557857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 209/403] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.558519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.609537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676e7c5-7700-4ea4-af03-ff062dbb56e7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.646214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.647098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.647225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.647419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.649619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] RBAC: Authorizing `identity:create_grant(project_id=738ecf5c1f5148d0b992e582fdf98c8d, group_id=5f9eedc79d5a4d338bf65c9e0f687ff4, role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.650841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.651045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a550f55-3373-4e50-8380-0eda80765e37 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.664911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 195/404] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Jul 27 19:55:09 2022] PUT /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:09.666145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687c57-ef93-4fb2-bd96-f44e50a7f30a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.666434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687c57-ef93-4fb2-bd96-f44e50a7f30a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.666710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687c57-ef93-4fb2-bd96-f44e50a7f30a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.666828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27360628-da18-42ca-ac00-454bb7716ce6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.672245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 196/405] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:09.672245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa687c57-ef93-4fb2-bd96-f44e50a7f30a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.727463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.728891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.729404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.730125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.733613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, group_id=5f9eedc79d5a4d338bf65c9e0f687ff4, role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.736287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fb5db-0829-463a-9088-9217d84e5c66 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.753092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:09.765018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 210/406] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Jul 27 19:55:09 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:09.765018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.765018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.765018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.801819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:09.803074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:09.846948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa687c57-ef93-4fb2-bd96-f44e50a7f30a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:09.897034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:09.897661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:09.897942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:09.898220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:09.898454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 211/407] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:09.899407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:09.900854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:09.902482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 197/408] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:09.908027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:09.936136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.023653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.053976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8cbba-a24e-4edc-b5a3-9ffaf4691178 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.068351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824fcc-0fe4-47d1-a02d-c3237fc73424 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.105325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.106310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.106772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.107224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.109122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.112499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.118386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 212/409] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:10.120132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.121471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.146984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.147970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.148442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.148903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.169704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:10.175773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:10.176669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:10.177508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:10.181773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 198/410] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:10.182724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.260818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9c978-6622-4197-adb2-a89584078487 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.310226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.311098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.311386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.311663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.315204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8dd370bebd048bd88eca9183841c3e3, group_id=5f9eedc79d5a4d338bf65c9e0f687ff4, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.317348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8853b-230c-44be-9d12-7c2a54acb508 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.327893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fe56b-ef6e-4276-bd94-503619f75b36 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.340063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 213/411] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Jul 27 19:55:10 2022] PUT /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:10.341147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b8ef-a8fa-4d43-b3d6-0810eb822c01 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.341444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b8ef-a8fa-4d43-b3d6-0810eb822c01 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.341720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5b8ef-a8fa-4d43-b3d6-0810eb822c01 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.343114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 214/412] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:10.351458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.351829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.352077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.382199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.383272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.383748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.384072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.384833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:10.385164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.386551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.398520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 199/413] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.404327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.418845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.438143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:10.439548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:10.473156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.566125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26953c7f-9775-428f-af1e-ab1e45288a39 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.591588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 215/414] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:10.616577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.621041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.621041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.621041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.621041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] PATH_INFO: `/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.621388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] RBAC: Authorizing `identity:get_group(group_id=331f387c2edf4fc7b8b6780d12b1eba3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.622407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.624296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 200/415] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:10.630111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.780039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61374797-030f-4d8e-9c60-39640d8f7664 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.806904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604080c1-b8ed-4d01-8d3e-8fc58421f8f3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:10.811448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.845323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.845804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.846338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.846614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] PATH_INFO: `/v3/projects/9e55cde5d6044829b509ee6bccad61b8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.849009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] RBAC: Authorizing `identity:get_project(project_id=9e55cde5d6044829b509ee6bccad61b8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:10.850461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:10.852336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 201/416] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/projects/9e55cde5d6044829b509ee6bccad61b8 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:10.858233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:10.876443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:10.877237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:10.877518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:10.877793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:10.882152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:10.894973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:10.901071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:10.901939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:10.902712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 216/417] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:10.913673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.024865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d7344-396b-48db-8b74-2a840adda448 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.083894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.084884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.085419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.085898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] PATH_INFO: `/v3/projects/9e55cde5d6044829b509ee6bccad61b8/groups/331f387c2edf4fc7b8b6780d12b1eba3/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.088063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] RBAC: Authorizing `identity:create_grant(project_id=9e55cde5d6044829b509ee6bccad61b8, group_id=331f387c2edf4fc7b8b6780d12b1eba3, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.089737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.107743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3c0e6-36e3-4a6b-b9b5-836d680965be None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.117374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 202/418] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Jul 27 19:55:10 2022] PUT /identity/v3/projects/9e55cde5d6044829b509ee6bccad61b8/groups/331f387c2edf4fc7b8b6780d12b1eba3/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:11.159918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.160606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.160886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.161181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.161911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:11.162208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.163727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.179168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 217/419] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:11.184618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.259897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.331642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fcbaa-66a5-4803-a471-6de62434e2ed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.368613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.369260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.369543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.369819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.442203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.443675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.489506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.549775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 203/420] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:11.555232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.640177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:11.698027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4d0c-6241-495c-b77f-c9d36677920a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.745965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.746973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.747483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.747957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.753693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.755388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.757713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 218/421] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:11.762830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:11.908939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ca768-68b5-45a7-9efe-88ab25785e6b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:11.947013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:11.947660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:11.947942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:11.948220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:11.949424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:11.950885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89593b-9648-4127-84f2-3868dbb32b61 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:11.952728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 204/422] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:11.960619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.109823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b6ee-91ff-4238-80b9-76dab35a1435 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.110481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b6ee-91ff-4238-80b9-76dab35a1435 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.110964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b6ee-91ff-4238-80b9-76dab35a1435 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.112789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 205/423] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:12.117509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.117797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.118072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.130095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa41158-78a1-4c5a-859a-6d601a8b728d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.137667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:12.138880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:12.147140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.177669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.198515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.199145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.199424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.199699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.200486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=cfc89a51451c403880880749d727c042, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:12.200794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.202249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392efe41-5c68-43c7-9a97-053841129c04 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.215053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 206/424] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:12.221033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 219/425] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:11 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=cfc89a51451c403880880749d727c042 => generated 611 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:12.239011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.278856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23c909-f55c-41b7-b18f-2570cecfd11b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.279144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23c909-f55c-41b7-b18f-2570cecfd11b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.279423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23c909-f55c-41b7-b18f-2570cecfd11b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.281006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 220/426] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:12.286681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.286966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.287243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.322106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:12.324536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:12.432309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4cfb0-2f23-435f-a1cb-010535f863dd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.457535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 221/427] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:12.482547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.483265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.483590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.483867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.484487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.495985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:12.502076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.502978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:12.503762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 207/428] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:12.510959 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.514609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.607760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e31793-fb4d-478e-80f3-1d987faf02ed None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.624574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2f785-997d-4172-9f57-11ab8953e83e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.632033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.650098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.651069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.651528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.651978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.663811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.664123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:12.664848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.665295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.665749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.666591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:12.667012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.668502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:12.675937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:12.677067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:12.678043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:12.679953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 222/429] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:12.681456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 208/430] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:12.686580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.693178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.759285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.801834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa5738-5092-436c-98f8-b004b2431de5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.802736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9115a2-38ff-48bb-83a0-0013b4708ffe None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:12.839299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.840382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.840714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:12.841151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.841360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:12.841644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:12.841920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.842223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:12.842648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:12.842953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.844359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.854959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 223/431] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:12.860806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:12.930796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:12.933361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea82c3-18dc-42da-90b9-21b551950c39 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:12.937442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:12.993889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6690-965a-4b31-9fd1-3d0a966b2dfd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.009501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.042626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.043258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.043535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.043806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.066896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 209/432] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:13.074546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c50a7-8cc9-4a2d-883f-de7c23f2dde7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.074843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c50a7-8cc9-4a2d-883f-de7c23f2dde7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.075118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c50a7-8cc9-4a2d-883f-de7c23f2dde7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.079124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 210/433] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:13.080055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c50a7-8cc9-4a2d-883f-de7c23f2dde7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.143795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.145320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.192837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.206993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c50a7-8cc9-4a2d-883f-de7c23f2dde7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.243629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.244276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.244566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.244843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] PATH_INFO: `/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.246078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] RBAC: Authorizing `identity:get_project(project_id=738ecf5c1f5148d0b992e582fdf98c8d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.247497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2234b0b7-47ef-4114-a514-c0eb29429fbb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.253744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 211/434] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/projects/738ecf5c1f5148d0b992e582fdf98c8d => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.253744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.253744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.253744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.260386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 224/435] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:55:13.264912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.271345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:13.272569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:13.299045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.343121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 212/436] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:13.344167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.384304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ba3ef-47fb-42d0-8aee-9176cf6f4e26 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.422093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.423073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.423549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.424443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.425838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.427440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.429358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 225/437] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.430125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.446527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.457940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7e30e-91b3-4379-955c-acc8e34dfe0b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.494983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.495861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.496145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.496431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.497592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.499033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.505087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 213/438] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.505087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.532863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.546354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110ac37-16fe-4f49-b38c-b62aaf1a9629 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.581325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.581957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.582242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.582519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.592766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:13.598457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:13.599337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 226/439] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:13.601105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.606091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794d204-e3c4-4581-a801-b926ddb73885 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.640436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.641071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.641487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.641764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.642556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.642863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.644291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ecf94-47d8-478b-8ea4-45cab9424913 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.650007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 214/440] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=738ecf5c1f5148d0b992e582fdf98c8d&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.651137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68ddb-b6ba-4645-b805-e22a1fe47d5c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.651418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68ddb-b6ba-4645-b805-e22a1fe47d5c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.651684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb68ddb-b6ba-4645-b805-e22a1fe47d5c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.657529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 215/441] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:13.657529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb68ddb-b6ba-4645-b805-e22a1fe47d5c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.682086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.718327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b3808-1b46-4b4a-bc50-83a7ff4cd4c7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.755967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.756721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.757024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.757306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.758502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.759965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73708c9-6f68-4334-aa27-ce09aa4da9b9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.766965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 227/442] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.766965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.766965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.766965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.776009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.783884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:13.785133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:13.786020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb68ddb-b6ba-4645-b805-e22a1fe47d5c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.823609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:13.824231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:13.824519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:13.824790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:13.825540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:13.825842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:13.827207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:13.838739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 216/443] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:13.839556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.866521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 228/444] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:13.867373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:13.951160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:13.963219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725e830-ea6b-4a63-bc03-143682c78f24 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:13.990847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f3299-12f1-4b41-9c70-f446a2a4d9ef None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.003303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.004022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.004197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.004478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.005311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=e8dd370bebd048bd88eca9183841c3e3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.005626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.007075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.013493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 217/445] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.015346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.040608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.043028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.043726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.044035 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.044283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.128275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.130050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.139181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dc5e-aa89-4f94-ae31-472b2e50f70d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.179036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.179949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.180388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.180664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.193756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:14.198973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:14.199898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:14.200692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 218/446] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:14.206765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.208784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.245866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 229/447] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:14.251932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.295123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.314893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887f33a-0bba-4c0d-9d8b-ab36f0ed61f4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.349070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a59cdb-7c34-4fbc-8608-560f733117ac None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.349493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.350101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.350485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.350648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.351354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.351653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.353079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.361133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 219/448] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:14.366636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.382806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.383445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.383726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.384002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] PATH_INFO: `/v3/projects/e8dd370bebd048bd88eca9183841c3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.385205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] RBAC: Authorizing `identity:get_project(project_id=e8dd370bebd048bd88eca9183841c3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.386659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.393020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 230/449] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/projects/e8dd370bebd048bd88eca9183841c3e3 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:14.394112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.427001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.467480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00797414-f792-46e7-a3d1-25527aabc4c4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.493144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532512f-83ac-4ac5-acca-23e53c260239 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.502302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.502862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.503146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.503421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.514929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.526182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.526806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.527084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.527354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] PATH_INFO: `/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.528497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] RBAC: Authorizing `identity:get_group(group_id=5f9eedc79d5a4d338bf65c9e0f687ff4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.529955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.531526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 231/450] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/groups/5f9eedc79d5a4d338bf65c9e0f687ff4 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:14.537183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.573726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.575172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.634170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf878d-e77b-49d9-b273-998e5989be8d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.636863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.669928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.670534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.670815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.671090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.671876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] RBAC: Adding query filter params (group.id=5f9eedc79d5a4d338bf65c9e0f687ff4, role.id=cfc89a51451c403880880749d727c042, scope.project.id=e8dd370bebd048bd88eca9183841c3e3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:14.672193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.672323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 220/451] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:55:14.673693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.677825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.681147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 232/452] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/role_assignments?group.id=5f9eedc79d5a4d338bf65c9e0f687ff4&scope.project.id=e8dd370bebd048bd88eca9183841c3e3&role.id=cfc89a51451c403880880749d727c042 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:14.724501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:14.784718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60985-eb81-4fa2-9ce6-3a5a6e8e142e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.822331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.822994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.823270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.823549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] PATH_INFO: `/v3/projects/9e55cde5d6044829b509ee6bccad61b8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.824736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] RBAC: Authorizing `identity:get_project(project_id=9e55cde5d6044829b509ee6bccad61b8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.826214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.827887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 221/453] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/projects/9e55cde5d6044829b509ee6bccad61b8 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:14.833662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:14.939298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921e78-7dfe-4259-bf33-0db3252125fe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:14.974110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:14.974751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:14.975037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:14.975308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] PATH_INFO: `/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:14.976448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] RBAC: Authorizing `identity:get_group(group_id=331f387c2edf4fc7b8b6780d12b1eba3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:14.977930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe69f3c-c66c-4189-aa30-7ae396b305f5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:14.979536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 233/454] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/groups/331f387c2edf4fc7b8b6780d12b1eba3 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:14.986386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:15.094179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37901-e989-4830-8ff0-282cc6574925 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:15.105370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:15.130482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:15.131099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:15.131379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:15.131653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:15.132460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] RBAC: Adding query filter params (group.id=331f387c2edf4fc7b8b6780d12b1eba3, role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e55cde5d6044829b509ee6bccad61b8) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:15.132766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:15.134229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f22013-e90b-473c-bc12-061d82242729 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:15.140177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 222/455] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Jul 27 19:55:14 2022] GET /identity/v3/role_assignments?group.id=331f387c2edf4fc7b8b6780d12b1eba3&scope.project.id=9e55cde5d6044829b509ee6bccad61b8&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:17.580384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cb63f-37ab-42a8-a27a-ee76491ffdc5 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.580908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cb63f-37ab-42a8-a27a-ee76491ffdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.580908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cb63f-37ab-42a8-a27a-ee76491ffdc5 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.582528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 234/456] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:17.587950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838cd78-e504-49b4-8f6e-86b37a2cab86 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.588365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838cd78-e504-49b4-8f6e-86b37a2cab86 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.588540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838cd78-e504-49b4-8f6e-86b37a2cab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.613935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3838cd78-e504-49b4-8f6e-86b37a2cab86 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:17.615122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3838cd78-e504-49b4-8f6e-86b37a2cab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:17.710853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 223/457] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:17.747391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cb63f-37ab-42a8-a27a-ee76491ffdc5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:17.799144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013892-a3ed-4dbd-ba75-e0cbb3de01f8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.800462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013892-a3ed-4dbd-ba75-e0cbb3de01f8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.804371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e013892-a3ed-4dbd-ba75-e0cbb3de01f8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.806006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 224/458] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:17.807583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:17.815774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd36e0-5424-4552-a22d-fbbe8cfa22b7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.816333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd36e0-5424-4552-a22d-fbbe8cfa22b7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.816812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd36e0-5424-4552-a22d-fbbe8cfa22b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.837001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4fbd36e0-5424-4552-a22d-fbbe8cfa22b7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:17.838874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4fbd36e0-5424-4552-a22d-fbbe8cfa22b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:17.917351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cb63f-37ab-42a8-a27a-ee76491ffdc5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:17.933747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 225/459] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:17.935080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd914-7b79-496f-abb2-9dd5524dc2f7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.935374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd914-7b79-496f-abb2-9dd5524dc2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.935651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd914-7b79-496f-abb2-9dd5524dc2f7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.941133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 226/460] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:17.942934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.943222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.943503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.956595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:17.957619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:17.958103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:17.958577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:17.962640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:17.963923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:17.972994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:17.975013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:17.990794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:17.991708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:17.992540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:17.992540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:17.992540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:17.992540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:17.993838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:17.995101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 235/461] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:17.996157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.052166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 227/462] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:18.053342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.127678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.144337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb10ed-7bdf-45f8-88c5-2c8c0defa501 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.191540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.192464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.192748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.193054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.193467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c254fe6-6878-4a55-bac8-8e70bd99deaf None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.208067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.210471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:18.225454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:18.226367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:18.227201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:18.227201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:18.227201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.227201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:18.229465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 236/463] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:18.230461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.238344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.239028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.239313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.239642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.240385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.240708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.242117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.259142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 228/464] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.260484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.291049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.341403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c1d6-c1fb-410f-9b22-e450f9e1ba13 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.374547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d780ea4-a917-4f40-956f-bb43abe00ea8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.378658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.379540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.380057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.380533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.390217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.395199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.396951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.412219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.412846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.413246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.413712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.414518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.414929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:18.415172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:18.416186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:18.417268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:18.417268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:18.417268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:18.417268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:18.418637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.419275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 237/465] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:18.420161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.436641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 229/466] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.437988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.528394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.530103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c7ba0-7514-4cdc-93f5-8487fc7a026d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.547652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975f26-1cba-4cd4-9594-ed670f63519d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.566852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.567727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.568012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.568290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.569068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.570512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.585879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.586420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.586921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.587038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.587780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:18.588038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.589466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca8da0-7511-4575-817d-a170c6c767a6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.593573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 238/467] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:18.597917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.611738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 230/468] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:18.614967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421b10-e41b-4c95-9183-506e03724fe5 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.616005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421b10-e41b-4c95-9183-506e03724fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.616775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421b10-e41b-4c95-9183-506e03724fe5 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.619672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 231/469] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:18.621287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34421b10-e41b-4c95-9183-506e03724fe5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.660196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.728371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451237a-6ff5-4abb-bc3c-14320cdfaedc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.770102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.770728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.771012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.771288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.771972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.774010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e36f9-6621-4b44-9216-37a04872aa98 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.774275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34421b10-e41b-4c95-9183-506e03724fe5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:18.801796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 239/470] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:18.803168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.803464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.803899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.817542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:18.818182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.818460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.818739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.820246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:18.821899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b42ef-7c24-4eda-aa14-fbb0846465be None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:18.826536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:18.827831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:18.850804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 232/471] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:18.851621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:18.857799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd6c2-0111-457a-9235-d8b41395821d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.858168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd6c2-0111-457a-9235-d8b41395821d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.858359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd6c2-0111-457a-9235-d8b41395821d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.860468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 233/472] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:18.873558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:18.874608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:18.874969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:18.903427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:18.904804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:18.932472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 240/473] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:18.959250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:18.998975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.008388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 234/474] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:19.043625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.088609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78aad32-7426-4b68-8e86-ff732b3369cf None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.132460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.133148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.133429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.133700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.147861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.149494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:19.166356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.167422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.168328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:19.168328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:19.168328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:19.168328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:19.168328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 241/475] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:19.173690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.193177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c98a0-929a-4045-a2e9-e5537dad02a9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.233410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.243252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.243970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.244316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.244723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.245354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.246734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.258704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.sql.core [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:19.258704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 19:55:19.258704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [parameters: {'id': '0eac94b5e064482690a31603e848ed6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 19:55:19.258704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80137) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:19.280053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:19.281008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application yield resource Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise self.value Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application session.commit() Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:55:19.282657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:55:19.285284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:55:19.287377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result.read() Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:55:19.299028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [parameters: {'id': '0eac94b5e064482690a31603e848ed6c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.301112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 19:55:19.302676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.304565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 235/476] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Jul 27 19:55:19 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:19.304701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.304850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.319669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00af1-1cf7-4000-8f1b-3512a5cfd226 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.359958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.360587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.360845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.361141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.361842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.362142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.363520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.393462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 242/477] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:19.395432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.506983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606910-cfb1-498d-9e52-34b3d58b7635 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.512656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26cb78-e1fe-428f-858d-af9564d8e59f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.550523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.551205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.551493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.551773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.552472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.553968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.575590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.576716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.577238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.577687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.590490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 243/478] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:19 2022] POST /identity/v3/users => generated 249 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:19.596006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7adad3d2-71a6-4626-a676-70070158e339 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:19.602244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:19.603165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 19:55:19.604004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:19.606539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 236/479] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:19.612647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:19.637371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:19.733037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22121b00-5a1c-4c73-a6b4-547f5fe4b5cd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:19.771435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:19.772364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:19.772788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:19.773298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:19.774197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:19.774649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:19.776210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804a2f9-1d85-46e3-b420-2d52d68b72dc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:19.787036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 244/480] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:21.117277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df576-3392-4ec1-ac81-fc0221cfad26 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.117845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df576-3392-4ec1-ac81-fc0221cfad26 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.117845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df576-3392-4ec1-ac81-fc0221cfad26 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.119569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 237/481] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:21.126242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cf6d0-df38-4f91-b24b-e8b5adb1cc91 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.126766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cf6d0-df38-4f91-b24b-e8b5adb1cc91 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.126766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cf6d0-df38-4f91-b24b-e8b5adb1cc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.155185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-950cf6d0-df38-4f91-b24b-e8b5adb1cc91 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:21.156370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-950cf6d0-df38-4f91-b24b-e8b5adb1cc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:21.222194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.231480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 245/482] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:21.271672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df576-3392-4ec1-ac81-fc0221cfad26 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.350651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.414206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7df576-3392-4ec1-ac81-fc0221cfad26 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.459017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.459665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.459943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.460212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.475260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:21.475429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe492eb-7c6f-4082-91c3-0ec46e117dd5 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.475657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe492eb-7c6f-4082-91c3-0ec46e117dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.475937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe492eb-7c6f-4082-91c3-0ec46e117dd5 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.477600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 246/483] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:21.480606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.481595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:21.482388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 238/484] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:21.483040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.483040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.483040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.488133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.502753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:21.504396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:21.637379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 247/485] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:21.670898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.694411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc6a8-8f6b-4a6e-865c-3d082d4bee7f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.720567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.745949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.746652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.746937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.747209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.747923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:21.748224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:21.749733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5c98c-33a2-48e5-ac63-c82f28318b16 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:21.758961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 239/486] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:21.760193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d701-14a7-4f4c-be62-d0fd732d2d8d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.760479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d701-14a7-4f4c-be62-d0fd732d2d8d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.760750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41d701-14a7-4f4c-be62-d0fd732d2d8d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.762227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 240/487] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:21.767582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.767864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.768141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.787266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:21.788443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:21.802979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3287986-97e6-4c66-9f97-42740633c8ca None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.831146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:21.842790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:21.843580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:21.843930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:21.844346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:21.855532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:21.861012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:21.862093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:21.863049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:21.863775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 248/488] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:21.864254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.869334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 241/489] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:21.870351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:21.978221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373e9dd-dad0-4977-9d79-46e9eda67bb7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:21.995600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4f098-ab5d-49f4-a8f5-bd587fbd7969 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.004338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.019018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.019598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.019883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.020166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.031210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.033141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.034229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.035261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.036108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.036542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.037425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.037865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.039351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:22.049367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.050398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.051417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:22.051417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:22.051417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.051417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:22.052259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 242/491] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.053310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.055259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 249/491] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.056844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.103045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.175756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773bb74-1e4f-4a75-93f3-eb8046fe0739 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.186449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd095f-e905-4f31-a1b1-2bb47f40ca8b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.214051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.214665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.214947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.215229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.215919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.216222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.217631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.224132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.225465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.225926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.226810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.229395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 250/492] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.230140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.243282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e8f88c44-6390-45c7-ba68-59ce1cb68764 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:22.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.251527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.252507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:22.259049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 243/493] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.260484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b7ff-ee38-4ada-a7f9-131688f6d2ea None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.260775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b7ff-ee38-4ada-a7f9-131688f6d2ea None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.261087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7b7ff-ee38-4ada-a7f9-131688f6d2ea None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.262488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 244/494] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:22.263071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b7ff-ee38-4ada-a7f9-131688f6d2ea None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.293396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.334125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.340611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7e625-f01c-4082-866e-6d5e98e50372 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.374707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7b7ff-ee38-4ada-a7f9-131688f6d2ea None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.376810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.377531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.377802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.378074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.388445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.389960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:22.405158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.406229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.408200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:22.408200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:22.408200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.408200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:22.411748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 251/495] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.412381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.416645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.416779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.417054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.417334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.425816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.427260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-51d8e624-7219-482d-8c09-155d608942b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:22.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.441145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:22.442355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 245/496] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.443124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.443292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.443570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.463133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:22.464339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:22.523723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b8b70-8aa6-4a55-af40-68f9ad21e253 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.532459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 246/497] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:22.533438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.552281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.559209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.559831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.560358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.560358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.561108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.561411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.562771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.570803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 252/498] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.571575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.584808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.636929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5d2e3-95c1-44c3-ab8f-4e9b876d1f83 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.670498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.670721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddd251-3a62-4bd0-952c-731e97f59976 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.671117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.671428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.671706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.673093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.673472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.674870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0e861-26c8-4f6d-9d3f-95d6c028727e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.686299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 247/499] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.687614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75922e78-3bc4-42e8-a004-b7df57648097 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.688020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75922e78-3bc4-42e8-a004-b7df57648097 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.688306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75922e78-3bc4-42e8-a004-b7df57648097 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.689570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 248/500] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:22.691921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75922e78-3bc4-42e8-a004-b7df57648097 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.710841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.711464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.711746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.712034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.712816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:22.713153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.715192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.754609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 253/501] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:22.755275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.783325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.807482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75922e78-3bc4-42e8-a004-b7df57648097 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.834591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:22.851413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.852313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.852776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.853283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.866186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:22.871533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.873667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 249/502] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.874500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:22.881445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cb0d-ec7b-4e80-962f-539c5531dc1a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:22.926207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:22.926874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.927360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.927837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.938500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:22.939981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-71fe345e-d304-48f9-9b04-2237a2e971e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:22.957140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:22.958183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:22.959251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:22.959251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:22.959251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:22.959251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:22.959251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 254/503] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:22.959831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:22.960121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:22.960397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:22.984526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:22.986881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:22.996476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d7e40-e5f2-47ec-9eeb-59b48779080b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.036407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.037373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.037848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.038143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.045928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.048465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.050386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:23.065089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.066371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:23.067746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:23.068374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 255/504] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:23.068374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.069170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 250/505] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.070891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.136333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.191649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10564b71-73dd-452a-9b23-1b3119022187 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.194503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9a150-3798-4aee-98b5-461c01161c62 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.232877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.233470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.233616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.233817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.234096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.234184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.234577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.235770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.236903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.237239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.238625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.248140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 251/506] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.249091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.311106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.313083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.319502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.366166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03691c7a-ff9e-4278-ae76-8ffe61714d21 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.374250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.410680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.411891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.412002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.412212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.413129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.413648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.415443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.432092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 252/507] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.433218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.445602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 256/508] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:23.446490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.549807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8083c97-e8dd-4713-8a2f-e158cb17fee3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.559141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347c39d-355c-47e6-943a-d3da7224c286 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.571308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.591686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.592827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.593108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.593393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.594255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.595062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.596499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.599897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.600890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.601397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.601688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.612795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.614327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:23.630037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.630941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.631781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:23.631781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:23.631781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.631781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:23.631781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 257/509] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.632047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.634874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.636662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 253/510] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.637907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.734276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a4f1-7a90-4eae-9195-d00137b70550 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.740378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667506-19cc-4e1e-b83e-bcdfd3ea8bb8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.768498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.769169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.769449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.769720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.775731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.776379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.776658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.776931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.781334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.783636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.784895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.786453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:23.797823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.798676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.799522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:23.799522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:23.799522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.799522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:23.799743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.800526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:23.801510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:23.802012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 258/511] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.802012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.803286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 254/512] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.803286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.841378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.859237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:23.902310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020bd485-794f-4576-a300-4e2d5f8b9bef None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.905407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d091-9b23-4d59-84c7-5b1e4adf6041 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:23.934813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.935429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.935707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.935981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.936662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:23.936985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.937571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:23.938758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:23.938908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.939027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:23.939298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:23.950020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:23.951189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 255/513] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:23.951466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:23.951787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:23.969612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:23.970621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 259/514] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:23.976032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.048778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.052998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77eeab-3e9c-4cfc-ad23-88d1cee494e4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.071013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072767-3196-4ce6-97c8-ac2141806923 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.078190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.089720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.090347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.090634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.090913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.110317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.110863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.111143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.111416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.112106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.112419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.113827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.125047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 260/515] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.126358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.158511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.159956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.223931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d2da4-b7d5-45a2-9a61-d30beadadc5b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.256029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.258471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.259086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.259362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.259634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.260422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.260727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.262202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.265536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 256/516] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:24.266732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.281662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 261/517] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.282502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.307295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.366412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd84337-efc9-448e-9074-38421da2415e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.381908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ade0fd-9d53-4a8b-a42c-6f67729b0c90 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.402729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.403341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.403620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.403892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.404558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.406026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.416250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.416866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.417171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.417444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.418135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.418443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.420121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.429616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 257/518] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:24 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:24.430522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.444408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 262/519] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:24.446074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.502232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.549770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.554774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6a2bc-8d00-4fc3-aa98-ea2b050851eb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.570197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc5950-98a3-47f6-b5a1-2eec5d7bcf3f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.594718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.595527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.595815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.596095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.604572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.606091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.611641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.612565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.612911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.613171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:24.617568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.618425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:24.619335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 258/520] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:24.619887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.625129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.630096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:24.647356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.648228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.649151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:24.649151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:24.649151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:24.649151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:24.649151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 263/521] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:24.649405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.737355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50efd68-ef8d-44d1-9504-7503258e2d08 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.753895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.755118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4786a8-c724-4927-80ef-be237b456196 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.776905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.777656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.778029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.778313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.792297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.792950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.793332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.793618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.798054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:24.803424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.805770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:24.819375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:24.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:24.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 264/522] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:24.821835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.858854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.861000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.930439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00852f3f-49db-4bf7-89e1-6525769c803e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:24.966120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 259/523] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:24.967390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:24.972211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:24.973314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:24.973314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:24.973606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:24.974380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:24.974690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:24.976161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:24.986743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.003261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 265/524] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.003261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.063981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.087448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988339a-b30c-4877-b0c8-f79e4841e55e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.122123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd0213-8557-4d6b-abbf-bf66b9e896b3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.123116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.123742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.124260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.124694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.125727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.125977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.127459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.140021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 260/525] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.140845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.170882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.171913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.173244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.173784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.174819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.175135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.176644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.196347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 266/526] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.197154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.226628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.260473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5539a86-2ddd-4ce1-a4ce-88ac3f503caf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.296111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.296753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.297067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.297990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.310509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.312467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.315237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8aad4-0643-4432-a87d-2743231de9ed None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:25.330653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.332638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:25.332638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:25.332638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:25.332638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:25.332854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.332922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 261/527] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.333040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.358181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.359038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.359480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.359814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.431477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.433391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.444197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6684f4-4ad1-45c0-9836-61a81c81fdd7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.470250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.479773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.480514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.480801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.481170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.488831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.490322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:25.502977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:25.503917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:25.504770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:25.509317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 262/528] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:25.510208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.546486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 267/529] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:25.546486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.572858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.622283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97e9dc-8659-49f7-b33f-fe37e357f003 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.658897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.659523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.659812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.660085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.660886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.661222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.662658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.668344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 263/530] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.669164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.692506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eae94f-93fc-4d19-843c-0adaaf562bd2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.732767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.748836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.749629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.750005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.750309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.751130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.751511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.753082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.765884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 268/531] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.771874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.795480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1777633-9000-4afa-a4e1-a64e5120dc4e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.839883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.840823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.841139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.841416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.843101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:25.843101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.843888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.865623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 264/532] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:25.866502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:25.898383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:25.912171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b9fe7-8c22-43e3-a9d1-f76e0c9142fe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.959007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:25.960025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:25.960950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:25.961484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:25.973383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc699a90-3556-4d1e-bdc8-a2fdcc6ef290 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:25.975469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:25.977940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:25.988605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:26.000523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:26.001707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:26.002562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:26.002562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:26.002562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:26.002562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:26.009061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 269/533] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:26.009061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.024665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.025588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.025970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.026338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.034869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.036673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:26.050702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:26.051756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:26.052780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:26.058717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 265/534] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:26.058717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.138833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cb443-54a0-4b51-806d-a62c1e39436c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.178729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.187584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81f136-d875-4e3b-a905-5471094fb086 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.190030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.191761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.192482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.192910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.193874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=d4256769bf5b4e81a15b985039cdbd80) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.194323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.196515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.205205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 270/535] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/role_assignments?user.id=d4256769bf5b4e81a15b985039cdbd80&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.206201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.243179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.244164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.244625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.245020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.245917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.249004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.249004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.272127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 266/536] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.282734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.323413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.364835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad60e1-7f56-40ee-84cd-21bda494cb21 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.407984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.408671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.408957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.410859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.411335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.411656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.413150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.428062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 271/537] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.437119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.456557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237f0d-21f3-4273-9af6-d223c64401a6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.496597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.497311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.497710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.498064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.506445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.508219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.514453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:26.526854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:26.527868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:26.528930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:26.536000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 267/538] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:26.537359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.649254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.660576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a9bc-2431-4608-af2f-dab93357a03d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.661469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7abbd-144c-4d2b-8202-4bd40e5a68ce None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.699186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.700123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.700436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.700746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.701149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.701377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.701656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.702053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.702458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=e5c57fe9a7f141458ac3d01bfa045695) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.702767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.703458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.704209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.704503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7812ce4-7b8c-4b89-8765-681217fec6be None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.706251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.715226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 272/539] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/role_assignments?user.id=e5c57fe9a7f141458ac3d01bfa045695&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.715226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d9c4-540b-4f0b-901b-1b3fe5a9a92d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.715226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d9c4-540b-4f0b-901b-1b3fe5a9a92d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.715226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d9c4-540b-4f0b-901b-1b3fe5a9a92d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.718641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 273/540] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:26.720290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.721245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.721245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.733525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 268/541] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:26.741631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:26.742855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:26.743165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.820949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 274/542] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:26.843712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:26.861152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04774-0454-416d-b6d7-747e83167d7e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:26.906394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:26.908570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:26.909291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:26.909804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:26.910842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6e73fb42abfa4d61b610555f657ec062) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:26.911386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:26.913097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:26.932359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:26.932603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 269/543] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/role_assignments?user.id=6e73fb42abfa4d61b610555f657ec062&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.018833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd802d6-2b07-4542-a088-4bf805fd5314 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.059929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.060608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.060892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.061199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.080627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:27.085914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:27.086899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:27.087690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:27.092945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 275/544] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:27.096181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.132259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.276676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239724-b968-4ddc-90c7-475aa87a2770 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.320936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.321907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.322360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.322803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.323698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.324225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.325867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.335786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 270/545] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.341003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.470675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7d5c-0124-48a3-b566-9b25e1254e3c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.473265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.514611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.515402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.515691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.516009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.531987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.533511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:27.552828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:27.553753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:27.554606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:27.554606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:27.554606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:27.554606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:27.554606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 276/546] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:27.559839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.677962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:27.708472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8bb4-f7e8-4ee3-96e9-9385a1b66da4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.751572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.752309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.752592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.752870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:27.753612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:27.753916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:27.755286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:27.774885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 271/547] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:27.780012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:27.941090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c96da1-b930-4b8b-845d-174ed4048f19 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:27.989840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:27.990482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:27.990766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:27.991041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.001248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.002730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-da60190a-a1be-4ef8-b3cd-cc41dc9a4263 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:28.018901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:28.019885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:28.021683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:28.024476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 277/548] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:28.025739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.057486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63008e9-a576-4a8e-b039-913afb387245 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.058106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63008e9-a576-4a8e-b039-913afb387245 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.058730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63008e9-a576-4a8e-b039-913afb387245 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.060524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 278/549] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:28.066607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.067627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.068097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.092122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:28.092122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:28.105095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.173169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 279/550] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 19:55:28.200061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.263763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd4277-bf7a-4430-b289-b1c0af4fc6a8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.331373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.333848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.335007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.335466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.337151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:28.337808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.340496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.347530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.350209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b51f15-e814-4107-a3ce-f29ada74ae59 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.376910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 272/551] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:28.390060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.406098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.406434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.406719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.406989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.429835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:28.438406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:28.439413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:28.440306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:28.440935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 280/552] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:28.445474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.455618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.584852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90303-9cd3-4cf0-867b-d2bac3f9ada7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.607906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e4660-dc73-4d8a-bb2f-7f7dcc7bb490 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.654121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.654751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.655032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.655312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.664086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.664818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.665134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.665410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.666136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:28.666440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.667832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.677356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 281/553] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:28.687288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.759055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.774451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.776068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.825403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:28.891628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 273/554] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:28.893359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4e598-7d18-4643-82ec-6a0b3d08a70b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:28.897242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:28.962269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:28.963403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:28.964041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:28.964518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:28.978117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:28.979891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:28.997444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:28.998382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:28.999236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:28.999236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:28.999236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:28.999236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:29.002503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 282/555] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.010010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.092340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.102953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec8c6-c7f1-46f0-9b2d-727ac7176c28 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.159830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.160516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.161078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.161366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.177325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.179735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.181324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6dcc9-ba51-4542-9065-a49c43afda0d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b6884174-807e-44e5-a2ec-de1ff3bb6e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:29.199932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.200939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.202928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:29.202928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:29.202928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:29.202928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:29.202928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 274/556] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:29.203303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.205848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343f0-3d23-4e46-b85e-2a88818470e8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.205848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343f0-3d23-4e46-b85e-2a88818470e8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.205848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343f0-3d23-4e46-b85e-2a88818470e8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.209102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 275/557] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:29.209914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271343f0-3d23-4e46-b85e-2a88818470e8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.223891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.224516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.224795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.225095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.226059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.226365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.227828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813e96-a0f3-4ee1-8a23-924a9deef994 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.241394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 283/558] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:29.242997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af169-dfae-49ad-83f8-2cbee892913d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.243356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af169-dfae-49ad-83f8-2cbee892913d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.243708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2af169-dfae-49ad-83f8-2cbee892913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.263048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bb2af169-dfae-49ad-83f8-2cbee892913d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:29.264576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bb2af169-dfae-49ad-83f8-2cbee892913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:29.334604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 284/559] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:29.335618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271343f0-3d23-4e46-b85e-2a88818470e8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.347238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d623e1-9708-4462-a3db-ae4a302acd08 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.347909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d623e1-9708-4462-a3db-ae4a302acd08 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.348532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d623e1-9708-4462-a3db-ae4a302acd08 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.350614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 285/560] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:29 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:29.351532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d623e1-9708-4462-a3db-ae4a302acd08 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.378133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.379232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.379594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.380034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.380735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.381095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.382445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fcc90-b2c2-4474-90fc-4fc5e04b5367 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.396038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 276/561] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:29.396209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.396558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.396840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.409602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.415933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:29.417159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:29.460052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d623e1-9708-4462-a3db-ae4a302acd08 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.471071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.483435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 277/562] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:29.484283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.506013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.506665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.506945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.507227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.514446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.516395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e58e6c15-7829-458c-8a16-013b17b1a9ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:29.529050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.530029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:29.530939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 286/563] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee2a7-7228-4da2-b2a7-bb7853f7c175 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee2a7-7228-4da2-b2a7-bb7853f7c175 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ee2a7-7228-4da2-b2a7-bb7853f7c175 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 287/564] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:29.536300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee2a7-7228-4da2-b2a7-bb7853f7c175 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.585024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd358511-7b3c-4e11-9f2f-11120297fd9d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.618868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.619512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.619795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.620076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.622705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.631122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:29.636051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.637045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.637866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:29.638487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ee2a7-7228-4da2-b2a7-bb7853f7c175 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.638902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 278/565] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.638902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.672228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.672860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.673161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.673440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.679858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.681344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:29.692852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.693953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 288/566] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.694831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.725788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.742440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c34d0-e0d6-4f6f-b276-47ce6b161c15 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.777200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.777766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.778046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.778318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.789272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2cfc32b1-9d04-4835-997d-9d71717c71de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:29.794284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:29.795358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:29.796210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:29.796850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e8cd5-c7c4-44c2-84a7-f92cf74d9378 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.796924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 279/567] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:29.796924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.796924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.797720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.815010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:29.816196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:29.830081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.830692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.830968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.831239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.832018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.832321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.833792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.850412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 289/568] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:29.851234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.870576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.887809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 280/569] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:29.888635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:29.938595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:29.951033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8d6c7-074d-4d60-8e42-1561059dd0ad None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.984826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:29.985465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:29.985742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:29.986037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:29.986770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:29.987069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:29.988429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:29.993600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485274-c53e-4907-960c-a678eb26a0b3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:29.996063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 290/570] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:29.996838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.029164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.029716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.029991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.030266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.031016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.031316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.032895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.044291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 281/571] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.045144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.096025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1072e9-f104-4869-9ee8-51494b400399 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.099996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.129291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.129969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.130251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.130523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.131227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.131524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.132896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.140529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 291/572] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.141424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.155579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.157824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e96e7-2a82-4eac-94e2-b93cf18231af None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.191650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.192536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.192809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.193119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.203925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.205388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:30.219974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.221175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 282/573] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.222494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.243504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03580945-a4e7-4320-ae8f-da9ca6d717f2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.276585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.277253 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.277535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.277814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.288812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:30.294222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.295304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 292/574] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.296154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.322007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.325885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726331e4-5661-466f-9700-84c94089fe55 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.358676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.359300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.360086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.360459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.365346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.370865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.372299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:30.386206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.387068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.387976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:30.387976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:30.387976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.387976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:30.387976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 283/575] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.388250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.399423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf811f-b4ef-4f5c-a104-7007fe83c301 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.433274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.433907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.434183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.434456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.435235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=bc49a64077c94d1194086179c518209e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.435539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.436989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.443110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 293/576] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/role_assignments?user.id=bc49a64077c94d1194086179c518209e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.443873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.493423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfcdf1-3216-45a1-a4fa-03b044107208 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.532005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.532666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.532946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.533252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.545150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.546561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:30.565149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.566153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.567028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:30.567028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:30.567028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:30.567028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:30.567028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 284/577] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.568228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.570795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.579450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3e6c9-dd4b-4d1c-a903-bb2211a34605 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.626521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.627141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.627560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.627833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.628518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.628817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.630220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.632563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.643711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 294/578] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.644466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.686609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4def3-10b7-4d26-966c-79a9a460927a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.723742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.724368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.724645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.725291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.726031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.726628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.728023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.740722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 285/579] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.740722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.753660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcbbfe-5321-42f4-8b72-36bbdfebc57a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.791091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.791682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.791970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.792290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.793010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.793322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.794736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.805732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.811709 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 295/580] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.811709 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.846630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c87b-9b91-40c2-b2a1-2743c5acc612 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.878372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:30.882409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.883037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.883313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.883620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.884339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:30.884646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.886070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.901024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 286/581] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:30.906653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:30.932549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adea0f9-309b-45bb-8101-6b6ab6fa6a42 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:30.971614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:30.972360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:30.972640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:30.973149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:30.980249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:30.981875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:30.994716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:30.995686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:30.996557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 296/582] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:30.997640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.015417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc2bf-ead0-42a3-81c9-3ea8cc2e90e6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.038889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.053136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.054355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.054355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.056099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.068523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.072607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:31.090150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.091080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 287/583] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.091942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.130069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a18da7-960b-45dd-a3c6-a6b29c7281b2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.147089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.166117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.166799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.167770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.167770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.175899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.177382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:31.189656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.190802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 297/584] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.191747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.207331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28e4f-0041-43bb-bfa2-5a18f515896a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.244712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.245431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.245712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.246016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.255125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.256574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:31.268160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.269131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:31.270048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 288/585] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.270649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.303155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24be4c-2c6c-4c30-a275-8a6135acb4bb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.314025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.339465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.340210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.340491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.340762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.341536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.341835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.343266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.360881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 298/586] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.360881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.388738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf1eff-582b-47b1-80a4-e3db0e5329b9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.394682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.425648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.426319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.426613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.426888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.427702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.428011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.429553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.446767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 289/587] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.446767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.467504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62055a90-63ba-4bed-a6c8-5d6377a2c0c4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.504260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.504928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.505238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.505551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.506345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.506649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.508063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.530716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 299/588] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.532209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.553389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.560615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e64aeb-8b7d-46c4-8fcb-4d3e0bce4152 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.599676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.600213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.600497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.600775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.601610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:31.601929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.603352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.623641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 290/589] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:31.624516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.639643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.648551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc769e3-d9b6-45f4-9f97-dd73d17b267f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.688409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.689390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.689893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.690307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.707681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=056e9b8fcfc84ee68a018f331e805582, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.709529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.723395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 300/590] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:31 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:31.724344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.741932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d80ae5-0a42-41fa-8932-6df740c8b58d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.795564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.796395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.797612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.798069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.808750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.811475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:31.829233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.830410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:31.831317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 291/591] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.831909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.851601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.865286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4645a17-4835-4f50-be2a-64240a932ee3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:31.915900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:31.917091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:31.917534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:31.917960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:31.931260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:31.932747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:31.935601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:31.949984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:31.951013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:31.951892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:31.951892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:31.951892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:31.951892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:31.956614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 301/592] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:31.957866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:31.967873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273916e-ea05-4d7f-aaca-65fcf5f3e205 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.006107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.007099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.007553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.007977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.021635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.023552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:32.040383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.041555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.042645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:32.042645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:32.042645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:32.042645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:32.044572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 292/593] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.049944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.065677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3484e59-ae1c-4a46-99fd-084d036d7c1e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.102393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.102918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.103200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.103475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.104489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.104608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.106095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.125192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 302/594] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.126176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.129536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.168058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e86908-a99f-418f-9d73-e1bb8deb274e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.186024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.209686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.210321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.210607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.210882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.211581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.212252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.214013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.232949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 293/595] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.234064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.238273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a58c2e-3eb9-458d-9283-310bd6bcd075 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.276114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.276740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.277059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.277339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.278036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.278341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.279846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.299411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 303/596] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.300315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.350595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8cbd6-6480-464f-90eb-9499a6448f15 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.381600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.398019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.398661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.398939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.399214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.412927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.414422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.429119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04b6dc-7c2a-4d45-9d93-8929825bc02c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:32.429883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.430761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.431655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:32.431655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:32.431655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:32.431655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:32.431655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 294/597] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.432339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.451623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.486806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.488094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.488605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.489297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.499071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.501705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:32.515605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.517329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:32.518211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 304/598] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.518698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.551830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200a271-1f4c-4568-b744-9a0421e7bc0c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.588122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.588751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.589057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.596973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.598432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5de3590c-6c4d-4828-b63b-a68900398ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:32.610487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:32.611419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:32.612872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 295/599] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:32.613511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcfc68-1189-4984-8278-5ea7f88a32c0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.613511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcfc68-1189-4984-8278-5ea7f88a32c0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.613989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcfc68-1189-4984-8278-5ea7f88a32c0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.615430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 296/600] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:32.619917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcfc68-1189-4984-8278-5ea7f88a32c0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.629504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3051-5562-4801-8e0a-e34563e55779 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.663209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.665070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.665952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.666140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.666401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.667341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.667457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.668767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.681621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 305/601] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.682760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.716552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.739702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcfc68-1189-4984-8278-5ea7f88a32c0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.775929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.776721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.777027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.777420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.778175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.778499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.780166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd030fff-6635-46a3-878e-05824695c1ad None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.783255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1a4d1-a4b9-41b7-9e1a-b1d4d377f341 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.801008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 297/602] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.802234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e466f-f884-4718-b35c-21095674be7a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.802520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e466f-f884-4718-b35c-21095674be7a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.802794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e466f-f884-4718-b35c-21095674be7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.819248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.819872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.820154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.820428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.821215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:32.821524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.821680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-466e466f-f884-4718-b35c-21095674be7a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:32.822911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-466e466f-f884-4718-b35c-21095674be7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:32.823005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:32.835892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 306/603] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:32.836606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.909143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 298/604] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:32.910320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e466f-f884-4718-b35c-21095674be7a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:32.942743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.944180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:32.952027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5953fd-b4b5-4b64-9df9-fbec403f7002 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:32.989236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:32.989878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:32.990214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:32.990498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:32.997547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:32.999004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:33.010695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.011593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:33.012482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 307/605] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:33.013663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.025781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e466f-f884-4718-b35c-21095674be7a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.060493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.061185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.061470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.061738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/d4256769bf5b4e81a15b985039cdbd80/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.077198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=d4256769bf5b4e81a15b985039cdbd80, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.079180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.095387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 299/606] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:32 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/d4256769bf5b4e81a15b985039cdbd80/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:33.096381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.148614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561c381-4570-4001-8de3-652d5a349b83 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.190028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.190432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.190686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.190962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/e5c57fe9a7f141458ac3d01bfa045695/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.207149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.210749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=e5c57fe9a7f141458ac3d01bfa045695, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.212118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.236440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb05a-170b-4b7c-a04f-80acae4e8259 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.237323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 308/607] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:33 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/e5c57fe9a7f141458ac3d01bfa045695/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:33.238381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.242832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.283704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.284783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.285308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.285796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.305083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:33.312400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.314528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:33.315423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:33.318737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 300/608] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:33.324937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.407268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547548c7-af85-4621-b01d-0927ed3964b1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.447960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.448925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.449539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.450098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.451060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.451591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.453062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6bbf6-483f-4058-9733-5f64750be44c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.453990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.467786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 309/609] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.474337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.492596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.493255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.493538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.493840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.494546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.494852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.496234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.503783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.508517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 301/610] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.513723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.548878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.591591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c911a-bd83-41b4-b8fc-9e7f8f706067 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.629196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.630175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.630632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.631076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6e73fb42abfa4d61b610555f657ec062/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.631478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc272d7-239b-439f-a7d5-123bee5fc614 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.647240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=6e73fb42abfa4d61b610555f657ec062, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.648995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.665793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 310/611] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:33 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6e73fb42abfa4d61b610555f657ec062/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:55:33.677184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80282162-05fd-41df-abbf-307165476742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.678044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80282162-05fd-41df-abbf-307165476742 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.678427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80282162-05fd-41df-abbf-307165476742 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.678893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80282162-05fd-41df-abbf-307165476742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.694560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80282162-05fd-41df-abbf-307165476742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.696300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80282162-05fd-41df-abbf-307165476742 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-80282162-05fd-41df-abbf-307165476742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:33.714515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:33.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:33.716317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:33.716317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:33.716317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:33.716317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:33.716317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 302/612] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:33.722243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.862032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8c18e-7f38-4b9b-a7dd-191e6757bbf2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.862358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8c18e-7f38-4b9b-a7dd-191e6757bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.862583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8c18e-7f38-4b9b-a7dd-191e6757bbf2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.863076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f85e32-2269-4773-9f26-132ec5da2600 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:33.864348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 303/613] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:33.873295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.873706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.874544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.885254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:33.908167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:33.909578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:33.935691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:33.936675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:33.937197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:33.939795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:33.940508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:33.940817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:33.942508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:33.956382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 311/614] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:33.961377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:33.976831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.000593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 304/615] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:34.032722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.100064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32373b4-7451-4162-afc6-9ceb3828d9cb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.137459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.138188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.145140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.145140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.148127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.149615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:34.162781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:34.163749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:34.164679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 312/616] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:34.172858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.211147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.243310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63527c-7b93-4793-b9be-8331649c58e6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.285971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760c6d-83a1-4e72-aa8c-80c685d1d52d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.324739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f151671-7091-4672-b490-5febb727143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.325386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151671-7091-4672-b490-5febb727143a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.325666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151671-7091-4672-b490-5febb727143a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.325939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f151671-7091-4672-b490-5febb727143a None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.326736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151671-7091-4672-b490-5febb727143a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:34.327046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151671-7091-4672-b490-5febb727143a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.328724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f151671-7091-4672-b490-5febb727143a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.335031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.335685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.335970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.336247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.349507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 313/617] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.354590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f151671-7091-4672-b490-5febb727143a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.362661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:34.368801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:34.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:34.370601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 305/618] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:34.376865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.427878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.537335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976563bd-2475-43b4-8284-09c9368d47bb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.539601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.551839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f151671-7091-4672-b490-5febb727143a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.590428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.591449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.592041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.592664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.602601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.603334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.603556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.603783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.604501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:34.604804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.606202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.606476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.608279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.624636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 306/619] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:34.629085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:34.630103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:34.631017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:34.631017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:34.631017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:34.631017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:34.633279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 314/620] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:34.635212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.638884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.794483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4a2da-6680-4281-a9cd-1aacf07318e3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.799776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:34.842480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19ca2d-7ddc-4c4c-9ebc-099e0389df66 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:34.863172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.863828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.864107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.864379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.881812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.883268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:34.907009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:34.908178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:34.909078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:34.909078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:34.909078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:34.909078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:34.909078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 307/621] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:34.913701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:34.914344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:34.914635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:34.914916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:34.915624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:34.915930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:34.917452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:34.917859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.937373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 315/622] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:34.943524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:34.950625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.061073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc8fbb-5f1f-4bc9-84a5-66b29903274c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.104073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.104573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.104858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.105167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.105869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.106175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.107553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.124812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 308/623] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.126898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.128114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205f070-7793-44d2-8de6-99700e6a31a9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.145936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.193708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.194443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.194648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.194923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.202152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.204423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:35.237377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:35.238716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:35.241299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 316/624] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:35.243888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da619a7-9a9a-4c1a-b1ff-172e277468f9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.245302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.283683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.284748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.285299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.285786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.294565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.296520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:35.310139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:35.311199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:35.312157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:35.319942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 309/625] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:35.321452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.389332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.403681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.430075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24351f-d9c9-4e58-8c8c-fb4d895cbf90 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.467355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.468368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.469044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.469706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.470879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.471410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.473075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.479087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c570-1c66-4362-83bb-6c71e7723378 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.499280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 310/626] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:35.502038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.577817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.578343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.578628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.578906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.579674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:35.579984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.585391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.613702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 317/627] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:35.617326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:35.644695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3395-028b-403f-9085-1fb441a3ea18 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.693532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.694209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.694493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.694769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.708569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.776360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.776602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840e8a-5b76-45cc-ac1d-e2642835fe4a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:35.777941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717394e1-6337-4f2c-b89a-5c2986bfdb61 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.803412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.815491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:35.816574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.818126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.818610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/bc49a64077c94d1194086179c518209e/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.837158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=bc49a64077c94d1194086179c518209e, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:35.839817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d3ef0-6b53-4252-975e-e097c458863a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:35.864996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 318/628] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:35 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/bc49a64077c94d1194086179c518209e/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:35.864996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92c2-383f-4d6a-8bc0-9990c3176fb1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.864996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92c2-383f-4d6a-8bc0-9990c3176fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.864996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92c2-383f-4d6a-8bc0-9990c3176fb1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.866203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 319/629] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:35.867477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5739ad-671f-4c0e-b31a-33c30b2bb521 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.867764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5739ad-671f-4c0e-b31a-33c30b2bb521 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.868039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5739ad-671f-4c0e-b31a-33c30b2bb521 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.869698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 320/630] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:35.872211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.872660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.873091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.896535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:35.897844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:35.903569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 311/631] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:55:35.904914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:35.905272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:35.905606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:35.930740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:35.932684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:35.975962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:35.989471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 321/632] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:35.990574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.024858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 312/633] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:36.025818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.123083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee2881-468b-45b3-898c-f6803e1654d8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.126189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.138677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d0008-1f11-4688-b6a5-bbc233a889da None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.159943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.160692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.160992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.161273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.172569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.174158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.175469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.176280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.176740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.177199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.188348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:36.191058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.192417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 322/634] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.193392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:36.196597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.197561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:36.198378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:36.201210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 313/635] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.202211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.229619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.306918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50ba2-a9f9-4ff8-ad2c-67f6e24276b3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.314914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83654e-4332-48a0-ab15-ab6b737da579 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.343206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.343868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.344144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.344432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.345135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.345438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.346812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.352291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.353022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.353314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.353592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.358535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 314/636] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:36.360070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.370039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b2966c68-8d61-4bac-81b8-120ea4b671a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:36.377069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.378003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 323/637] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.382603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a122c5-a68c-4efc-b13b-86ab8e398aa9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.383353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a122c5-a68c-4efc-b13b-86ab8e398aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.383353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a122c5-a68c-4efc-b13b-86ab8e398aa9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.383353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 324/638] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:36.383353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a122c5-a68c-4efc-b13b-86ab8e398aa9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.407459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.460094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.478661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916325b-2cee-4708-a3fd-c8b3683f7dc7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.501418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a122c5-a68c-4efc-b13b-86ab8e398aa9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.518536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.519154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.519430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.519700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.520414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.520717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.522096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.534999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 315/639] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:36.534999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.541541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.542188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.542471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.542742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.550689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.552144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a4fa745b-cc44-43ab-ae8d-edfdc0a52a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:36.565473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.566377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:36.567578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:36.571825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 325/640] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.572913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.573222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.573494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.591687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:36.592877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:36.644187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc9b25-02e8-487c-849c-9e09bad75576 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.662629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 326/641] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:36.673878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.680922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.682189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.682653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.683074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.683928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.684412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.684669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.687106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.696194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 316/642] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:36.697039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.712326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.789569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68c172-b9f4-4f4b-a542-93536f66e52e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.817860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983cec-2ecd-400b-ba7c-0142a846a2f9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.825402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.826044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.826329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.826605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.836707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.838316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.853064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:36.853674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:36.853948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:36.854217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:36.854407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:36.855322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:36.856249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:36.856249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:36.856249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:36.856249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:36.856472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:36.856472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:36.857059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:36.861442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 327/643] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:36.862604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.877881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 317/644] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:36.877881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:36.934179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.947043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:36.982371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0f00-193b-4012-b456-dfe1d96c0e30 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:36.986421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4ed8b-6a1f-4044-beae-1f2113cea036 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.022012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.022650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.022771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.022955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.023220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.023322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.023578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.023855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.035698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:37.036048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.037500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:37.041201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.042125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:37.043084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 328/645] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.043807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:37.058794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.059730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.060596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:37.060596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:37.060596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.060596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:37.060596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 318/646] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.060876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.172719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33208670-6cd7-4194-95c9-51d1b0bda722 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.175101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce551db-6681-4c61-ae06-b76d0549983b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.193827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.195801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.211653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.212387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.212515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.212803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.213173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.213420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.213711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.213991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.214084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.214355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.214680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.214989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.215841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.216351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.229342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 319/647] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.229342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.229592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 329/648] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.229592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.358656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f917f-b155-45bc-8e9b-1d2962f7b361 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.363100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ba427-a9de-4baf-ab28-76beaf28dc2b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.404454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.405463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.405463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.405665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.408427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.408427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.408797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.412217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.412850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.413180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.413542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.414248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.414558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.415919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.421231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 320/649] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.423115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.429139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 330/650] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.429623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.477078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.485853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.554371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbd40e-1d10-482e-b6ce-298c6ab20107 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.559070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb592ee-ad7f-49d7-99f0-694a2d47fd0c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.595657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.596622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.597088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.597488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.605499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.606148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.606431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.606702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.610718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.612213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.615654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.617183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:37.628081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.629044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.629950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:37.629950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:37.629950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:37.629950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:37.630180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.631024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 321/652] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.631944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.634030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 331/652] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.637981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.737584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.742849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561506-de44-44fb-85ff-e6a323e2829b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.746231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43eb575-cc78-4ba0-9eb0-6aeedd9fb1de None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.750456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:37.779418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.780100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.780383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.780711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.786972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0848-a410-48f9-8180-55b05626581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.787863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0848-a410-48f9-8180-55b05626581f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.788364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0848-a410-48f9-8180-55b05626581f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.788942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0848-a410-48f9-8180-55b05626581f None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.789870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0848-a410-48f9-8180-55b05626581f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.789977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.790486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0848-a410-48f9-8180-55b05626581f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.791390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.792789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0848-a410-48f9-8180-55b05626581f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:37.807364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:37.808622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:55:37.810107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:37.813267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 322/653] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:37.821037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.821420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 332/654] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.822712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0848-a410-48f9-8180-55b05626581f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:37.932819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693a44-76a2-4b63-9de6-7dca545af21c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.954022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0848-a410-48f9-8180-55b05626581f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:37.974021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.974653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.975461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.975752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.976849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.977190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.979490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.991158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:37.991712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:37.991990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:37.992262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:37.993096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:37.993406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:37.995300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8dd40-71c0-4ba3-840a-c8da75ec34e1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:37.997540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 323/655] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:37.998567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.018705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 333/656] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.020131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ce28-01a1-4811-8aa5-81167677febc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.020542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ce28-01a1-4811-8aa5-81167677febc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.021001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ce28-01a1-4811-8aa5-81167677febc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.022362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 334/657] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:38.026838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03ce28-01a1-4811-8aa5-81167677febc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.028242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.038904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.114989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fee-99c7-4a45-9bfd-5b6f0a3fa206 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.145477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03ce28-01a1-4811-8aa5-81167677febc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.156391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.157707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.158330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.158898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.168439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.170389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:38.184306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.185354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 19:55:38.186511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:38.190788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 324/658] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.191767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.199594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.200248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.200532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.200808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.275944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.277342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81420f1f-1887-4e8f-bafd-704842bc26e3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.285740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.287366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.304682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48bfaeb-1ff5-4bc8-8370-7b107a361c30 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.340056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.340676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.340991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.341272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.405126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 335/659] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:38.405126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.405126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.405126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.418714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.420158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.421554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:38.423869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:38.498549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 336/660] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:38.499639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.520116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.540549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 325/661] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:38.541140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.550337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.612478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f071-62e6-4627-8d3c-e49394a0b370 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.647787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ce195-5a3d-4553-a072-9e647a92f335 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.647997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.648583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.648862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.649168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.649948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:38.650252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.651681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.673276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 337/662] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:38.673276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.688372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.689046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.689333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.689612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.701251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.702782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:38.717833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.720197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:38.720197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:38.720197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:38.720197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:38.720197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 326/663] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.721036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.756127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.778427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:38.786482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90a06-f046-4c33-9326-0448cab39eea None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.823550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.824215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.824673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.824889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.827298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f888-cc83-4ce6-a47a-4a44660645a1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.836176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:38.841188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.842355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:38.843195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 338/664] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.844585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.874748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.876221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.876639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.877062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:38.888196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:38.889681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:38.904264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:38.905358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:38.906753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:38.906753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:38.906753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:38.906753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:38.906753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 327/665] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:38.907041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:38.960095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e43c1-7787-4422-9f7d-a05365882b18 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:38.998177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:38.999063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:38.999491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:38.999948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.007480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.020203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731463dc-9a0e-43fa-9d63-c3892b12dbd4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.028068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.057220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.057846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.058132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.058411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.059105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.059414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.060800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.074056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 328/666] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.074815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.097836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.099774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.183307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8a55d-a839-443d-958c-38503f565825 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.220793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353716-080e-4e21-ae8d-048099912834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.221711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353716-080e-4e21-ae8d-048099912834 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.221989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353716-080e-4e21-ae8d-048099912834 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.222261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23353716-080e-4e21-ae8d-048099912834 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.222981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23353716-080e-4e21-ae8d-048099912834 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.223285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23353716-080e-4e21-ae8d-048099912834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.224669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23353716-080e-4e21-ae8d-048099912834 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.231514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 339/667] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:39.231514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.235217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 329/668] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.239704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353716-080e-4e21-ae8d-048099912834 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.252352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.291483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.344884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49e2d-511f-49a2-af95-dcc99f35972e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.352047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23353716-080e-4e21-ae8d-048099912834 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.383383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.384004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.384292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.384592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.385272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.385594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.386947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.387429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.388096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.388377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.388639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.396115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.397583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.403282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 340/669] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.403282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:39.414390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:39.415309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:39.416178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 330/670] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:39.416653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.490671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.509516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff488a4-5d3f-4724-8837-5c158497fcf8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.522533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fee5e4-ab22-4564-a0ea-1a3e093839dd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.525573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.543782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.544417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.544695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.544993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.555275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.556718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.558948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.559594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.559876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.560151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.571779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.573242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:39.573376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:39.574242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:39.575131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:39.575131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:39.575131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.575131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:39.577118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 341/671] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:39.578088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:39.588894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:39.590047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 331/672] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:39.591297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.682417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2f83d-f70b-46d9-b493-682192c59c93 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.697206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf284ad2-7209-4842-a4d8-81722c9adcee None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.716284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.716927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.717237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.717511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.720035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.724997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.726447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.731853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.732462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.732740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.733041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.733790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.734094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.736222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f0b541c4-7a93-4bbd-85b4-4493a48b8a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:39.738973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:39.740033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:39.740937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 342/673] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:39.741772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3dc9-fe9f-43ea-8b40-b7eec3c4b20d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.741772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3dc9-fe9f-43ea-8b40-b7eec3c4b20d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.741905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad3dc9-fe9f-43ea-8b40-b7eec3c4b20d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.743191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 343/674] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:39.743712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad3dc9-fe9f-43ea-8b40-b7eec3c4b20d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.751851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 332/675] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.752672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.773105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.848047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad3dc9-fe9f-43ea-8b40-b7eec3c4b20d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.853555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81a5c8-7df9-4610-a5b8-d0d5ea35973c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:39.881881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.882508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.882787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.883062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.883770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.884072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.885501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.889642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:39.890253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.890610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.891501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.892265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:39.892563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:39.893974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebf8a6-7111-4758-9668-699438192dc3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:39.900201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 344/676] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.901048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:39.905450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 333/677] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:39.906614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:39.906895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:39.907172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:39.924127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:39.925339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:39.961732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.988506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:39.994524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 334/678] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:39.995348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.006968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60ecd6-3f54-4a1c-bbdb-23068bd93092 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.041045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.041664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.041945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.042227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.042977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.043291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.044719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.059577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 345/679] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.060420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.107059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e7d14-9600-443f-9e4b-b6ef2628337b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.148791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.149439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.149721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.149996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.150791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=d4256769bf5b4e81a15b985039cdbd80) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.151102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.152546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.159948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 335/680] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:39 2022] GET /identity/v3/role_assignments?user.id=d4256769bf5b4e81a15b985039cdbd80&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.160944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.184710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3859e0-9a6f-49bc-816c-56fe340ec167 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.205685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.222649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.223397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.223679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.223964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.231262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.232826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:40.246855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:40.247837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:40.248749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 346/681] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:40.249283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.266512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.293601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443e5eb-a600-43f4-a2b7-be885cfe9454 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.336383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684e796-d999-4598-9132-52594058d9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.337067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684e796-d999-4598-9132-52594058d9ea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.337344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684e796-d999-4598-9132-52594058d9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.337615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684e796-d999-4598-9132-52594058d9ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.345977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684e796-d999-4598-9132-52594058d9ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.347645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684e796-d999-4598-9132-52594058d9ea None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3684e796-d999-4598-9132-52594058d9ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:40.360540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:40.361845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 336/682] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:40.363095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684e796-d999-4598-9132-52594058d9ea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.369353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07821278-fa14-4bc9-b2f5-5b83aa07a2a2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.409651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.410319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.410605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.410881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.422103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:40.427896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:40.428844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:40.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 347/683] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:40.430489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.477239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.484096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684e796-d999-4598-9132-52594058d9ea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.513622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.523580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.524629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.524913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.525229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.526395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=e5c57fe9a7f141458ac3d01bfa045695) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.526717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.528485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.545008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 337/684] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/role_assignments?user.id=e5c57fe9a7f141458ac3d01bfa045695&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.545008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.551207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea404aa4-4d26-496e-ae87-df6cc01a96e1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.591614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.592400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.592784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.593187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.596065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.596501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.598141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.616237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 348/685] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.617373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.685940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d9772-0c59-461f-b0d1-0d076140f2ad None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.730662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.731216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.731496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.731840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.732741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.732906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.734386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.746855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3332c-af74-4cd1-8ff4-f240087096aa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.783861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.787671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.788413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.788852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.789959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.790724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.791040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.793225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.805108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 349/686] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.806345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 338/687] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.880144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:40.900057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:40.916288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904466-f0a3-47f7-88fe-c3361a8f348b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:40.951835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:40.952602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:40.953036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:40.953599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:40.954546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6e73fb42abfa4d61b610555f657ec062) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:40.954956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:40.959546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:40.967070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 350/688] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/role_assignments?user.id=6e73fb42abfa4d61b610555f657ec062&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:40.968019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.015217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c3108-76fc-45fb-ba98-b87d61547d3a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.091742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.096416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.096954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.097428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.102361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.120596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.122102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:41.154457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.155436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.157337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:41.157337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:41.157337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.157337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:41.157337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 339/689] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:41.163267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.164595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224205-98f3-450c-b69e-53a4fba23679 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.203404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.204393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.204910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.205525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.257218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.289704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.291752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.298933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f645b-2a99-41e4-a24e-b38481402bbb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.338911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.339546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.339840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.340091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.340794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.341131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.342604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.354319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.362130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 340/690] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:41.368456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.417018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 351/691] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:41.419608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.575327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acb71f-430e-4ab4-9acf-3f89b81a5173 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.575756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea25bb-348f-4fb0-949c-8419a1fe65de None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.612617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.614570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.615290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.615977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.629878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.631686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.642766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:41.649612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.650572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.651461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:41.651461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:41.651461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:41.651461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:41.658660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 352/692] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:41.660111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.662503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.663156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.663435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.663708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.673418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.689839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.691308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:41.717090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:41.718125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:41.720008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 341/693] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:41.724192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.787297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a916e-9fd4-4df2-9a0a-8c26629e4bcc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.852643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.854091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.854772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.856468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.857402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.858101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.861132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:41.888496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 353/694] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:41.890774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:41.908458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb0410-39dd-4bac-9fc6-0c85e062eb3b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:41.972683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:41.973798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:41.974390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:41.974881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:41.976110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:41.976641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:41.978103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:41.978900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.014992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358800e-40bd-4ac3-bb49-edfa6c51624c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.015382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 342/695] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:42.020402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.085212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.086989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.087664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.089265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.099541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.104689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:42.131434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.132402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:42.133328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:42.140524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.141343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 354/696] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:42.143451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.247685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22306d1b-e046-4caf-bbfc-95bddca43604 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.295195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fcf9c-d438-40bc-a4f7-2a6509c53ac7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.316380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.317026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.317562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.317834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.329407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.342938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.345280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.346442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.346927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.348175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.350027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.351744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.384165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 355/697] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:42.386421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.462659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.465286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b2ee0-1ca4-4731-a089-f5569a4828b5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.540426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.554611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad4670-6ac4-4170-ac91-0d4cf8a81c59 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.615571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.616485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 343/698] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 19:55:42.616875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.617458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.617819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895bffa-7fd6-4121-8b53-8fc154d92c43 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.618097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895bffa-7fd6-4121-8b53-8fc154d92c43 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.618349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.618450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895bffa-7fd6-4121-8b53-8fc154d92c43 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.619449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=bc49a64077c94d1194086179c518209e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:42.619834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 344/699] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:42.620221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.629011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.629652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 356/700] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/role_assignments?user.id=bc49a64077c94d1194086179c518209e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:42.630632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.631069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679da2-2038-4316-8044-0e366778bffe None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.631501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679da2-2038-4316-8044-0e366778bffe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.631920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679da2-2038-4316-8044-0e366778bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.672327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6a679da2-2038-4316-8044-0e366778bffe None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:42.673585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6a679da2-2038-4316-8044-0e366778bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:42.734787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.817621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 345/701] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:42.830772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c7bdf-e490-4948-a3ed-ca7662ab6004 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:42.842300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a679da2-2038-4316-8044-0e366778bffe None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.874284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:42.875745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:42.876595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:42.877120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:42.890660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:42.892491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:42.912069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:42.913020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:42.914006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:42.914006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:42.914006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:42.914006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:42.916498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 357/702] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:42.922556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:42.945773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:42.975936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a679da2-2038-4316-8044-0e366778bffe None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.021490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.023520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.023992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.024477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.041401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.043151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.057659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb9283-d474-450e-a15a-206d7215e0a9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.059288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f59425dc-2899-4075-aa31-a0f14d036e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:43.067869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.068798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.069730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:43.069730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:43.069730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:43.069730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:43.069730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 346/703] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:43.080986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ed65-65f5-4b3b-a004-d35e86cd6403 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.080986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ed65-65f5-4b3b-a004-d35e86cd6403 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.080986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513ed65-65f5-4b3b-a004-d35e86cd6403 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.080986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 347/704] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:43.080986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513ed65-65f5-4b3b-a004-d35e86cd6403 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.108893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.110797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.111429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.111717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.112425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.112999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.115083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724769c8-ac7b-442c-9b44-42b7c02098f8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.131856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 358/705] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:43.134030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.134314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.134589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.158140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:43.160053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:43.214811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513ed65-65f5-4b3b-a004-d35e86cd6403 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.252173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.255658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 359/706] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:43.256581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.267721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.268407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.268688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.269036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.269798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.270106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.271615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.295565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 348/707] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.302078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.368438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.379576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa78e8-269e-4ce9-b54d-47e203fe7c5b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.416893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.417997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.418595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.419116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.427785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.428677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085af11f-9e22-46d8-8a9f-4b874f266db4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.429764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:43.443921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.445093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:43.446164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:43.448109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 360/708] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.449076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.472134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.472676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.473206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.473326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.473921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.475344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.502656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 349/709] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:55:43 2022] POST /identity/v3/users => generated 254 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:43.502656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.556073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.601762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be005cfc-f87b-46fc-8c15-3dd474b9f9e9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.641906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479591c-5e21-4270-937f-7d21eb761162 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.648449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.649327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.649731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.650102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.650839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.652330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c9ef038d-6409-4b0d-bac1-b3fb142fa7f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:43.664713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:43.665916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:43.666762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:43.667538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 361/710] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:43.672993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad4f9b-7cbb-46b5-afd3-64250dbf4281 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.672993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad4f9b-7cbb-46b5-afd3-64250dbf4281 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.672993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad4f9b-7cbb-46b5-afd3-64250dbf4281 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.675490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 362/711] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:43.676396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad4f9b-7cbb-46b5-afd3-64250dbf4281 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.684902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.685618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.685903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.686182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.686956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.687357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.688924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f3112-840b-4856-9692-338a5ad839d9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.697197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.705841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 350/712] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.713217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.713924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.714737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.742060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:43.744095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:43.814646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad4f9b-7cbb-46b5-afd3-64250dbf4281 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.833027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 351/713] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:43.834783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.862285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.862923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.863277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.864145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.864842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.865187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:43.866998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:43.874257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 363/714] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:43.875589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:43.880472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.959698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbc1c3-fbd5-4d40-8928-224c8ace8ac2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:43.993544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:43.995855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:43.996578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:43.996864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:43.997163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:43.998062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:43.998385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.000542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.010854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3058-0940-452f-be30-888a006546c8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.013163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 352/715] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:44.015298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:44.054239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.055633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.056130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.056604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] PATH_INFO: `/v3/services/network` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.057532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.059266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:44.077728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:44.079667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:44.081252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:44.084611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 364/716] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:43 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:44.092866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:44.239055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff699-18d1-4ec5-a75c-aaa7fcbca3dd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.279718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.280289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.280574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.280851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.282173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] RBAC: Adding query filter params (name=network) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:44.282520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.284047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.289810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 365/717] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:44.293479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:44.295324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:44.297747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49ce2f-091a-490d-af28-1162e6fc98d2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.309646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:44.387737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.388711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.388930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.389320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.390119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:44.391093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.392742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218f49-d374-4c1a-bf67-373fb24d741b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.407748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 353/718] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:44.416385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088379bf-7ea2-40b2-87a4-d369b593286f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:44.456334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:44.457268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:44.457698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:44.458273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:44.459106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] RBAC: Adding query filter params (type=network) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:44.459573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:44.461120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe34f0-b86f-4981-925f-343beebef871 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:44.467764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 366/719] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:45.016435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce17df-6c2c-4973-a94f-0f535e707898 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.016993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce17df-6c2c-4973-a94f-0f535e707898 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.016993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce17df-6c2c-4973-a94f-0f535e707898 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.018597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 354/720] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:45.024543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.024863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.025128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.049974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:45.051177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:45.155522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 367/721] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:45.211589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce17df-6c2c-4973-a94f-0f535e707898 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:45.350437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce17df-6c2c-4973-a94f-0f535e707898 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:45.394256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:45.394956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.395242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.395514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.411320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:45.416941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:45.417895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:45.419653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 355/722] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:45.428058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:45.430265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:45.558742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcd4d9-40bf-4c32-bc14-50971bf7b2f7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:45.612073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:45.612615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.612898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.613198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.613963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:45.614272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:45.615684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:45.629393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 368/723] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:45.634935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:45.666661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:45.766163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f27e4-99e1-4a73-89f4-54fe335b6a10 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:45.812028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:45.813285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:45.813562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:45.813832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:45.829943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:45.831433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:45.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:45.853806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:45.855969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:45.855969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:45.855969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:45.855969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:45.855969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 356/724] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:45.866155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:45.930993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:45.976290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c55a06-4daa-4f30-977f-6e24eaab7cba None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.016668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.017380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.017657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.017926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.018620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.018917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.020282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67943219-d134-4293-85f0-b051c35cbe90 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.035072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 369/725] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.040796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.123012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a741d-e58e-4080-ae12-ebf4f7d6e7fe None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.123298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a741d-e58e-4080-ae12-ebf4f7d6e7fe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.123481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a741d-e58e-4080-ae12-ebf4f7d6e7fe None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.124855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 370/726] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:46.130171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.130700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.131182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.153377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.167665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:46.168853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:46.197094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff42b10-27de-4709-b234-3f3ff2ff671e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.258430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 371/727] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:46.281560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.309711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.310754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.311241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.311716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.330137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.331926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.345329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:46.363212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.364189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:46.365190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:46.369064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 357/728] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.370386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.395186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e5ecf-2790-469f-bf6e-2c4efc3b2e63 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.433321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.434733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.435220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.435699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.455120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8e41d25e-001d-4b08-a6c6-53b048385d7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:46.460493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.461525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:46.462579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:46.465102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 372/729] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.466774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf6559-30f5-4fda-ae9f-3626329953af None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.467462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf6559-30f5-4fda-ae9f-3626329953af None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.468109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf6559-30f5-4fda-ae9f-3626329953af None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.471292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 373/730] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:46.472152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf6559-30f5-4fda-ae9f-3626329953af None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.489035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840ed33-ae18-43be-890b-99da434782bb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.523789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.524601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.524952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.525345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.526276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.526688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.528204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c882d-8cae-4390-ae95-9ea569109b00 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.537495 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.551367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 358/731] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.552853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.553351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.553818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.573351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:46.574670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:46.584233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf6559-30f5-4fda-ae9f-3626329953af None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.600024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.622864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.623761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.624207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.624630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.626068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:46.626521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.628052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.637209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 374/732] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:46.638211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.647768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 359/733] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:46.654150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.796039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763aeef-703d-415c-a5e0-48833f51dd48 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.830358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:46.850679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.850679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.850679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.850679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.860525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:46.862105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:46.877882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:46.878831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:46.879762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:46.879762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:46.879762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:46.879762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:46.883927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 360/734] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:46.884872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:46.898354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80306b10-3806-4dec-bb63-ac4e4e0c69ab None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:46.940602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:46.941247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:46.941528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:46.941803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:46.999123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.009762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7a17-9f62-432d-8538-ad4f32b5ab9c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.023905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.025408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c3306-f8f7-4fea-8b7d-18eca28bda7b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.056139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.057457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.057888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.058309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.074067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:47.080626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.081582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:47.082397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 361/735] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.083077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.123317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.138990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 375/736] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 19:55:47.140237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10424e3c-98fd-4c2c-a171-30a8eaf4cbd9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.140704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10424e3c-98fd-4c2c-a171-30a8eaf4cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.141067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10424e3c-98fd-4c2c-a171-30a8eaf4cbd9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.142458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 376/737] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:47.146286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10424e3c-98fd-4c2c-a171-30a8eaf4cbd9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.200842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1581be-faf1-44cb-a8cb-7e77f6d6ecdd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.238816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.239427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.239708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.239986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.240679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.241006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.242380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.243969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.256016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 362/738] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.256808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.268337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10424e3c-98fd-4c2c-a171-30a8eaf4cbd9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.302266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.302878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.303159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.303435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.304146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.304451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.306138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af164060-b1d6-49ad-b8e9-1c464549d175 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.314074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 377/739] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.315239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5765b3-94a3-4c5a-9be2-996bd5868d4c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.315523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5765b3-94a3-4c5a-9be2-996bd5868d4c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.315798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5765b3-94a3-4c5a-9be2-996bd5868d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.333235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4a5765b3-94a3-4c5a-9be2-996bd5868d4c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:47.334435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4a5765b3-94a3-4c5a-9be2-996bd5868d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:47.350503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.356424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980c8aa-93ad-41c2-a799-d4664570d0f9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.390243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.391044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.391341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.391615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.399410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 378/740] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:47.400490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda7c0-4e01-4665-80de-c15722a4d5b7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.400787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda7c0-4e01-4665-80de-c15722a4d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.401090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda7c0-4e01-4665-80de-c15722a4d5b7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.402052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.405141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 379/741] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:47.405141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bda7c0-4e01-4665-80de-c15722a4d5b7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.405362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:47.423132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.424106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.425019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:47.425019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:47.425019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.425019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:47.425019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 363/742] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.425320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.474341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.510524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bda7c0-4e01-4665-80de-c15722a4d5b7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.523212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb2f66-3bcb-449e-b087-036390bee4aa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.543346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.543982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.544264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.544541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.552182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.553655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.555830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.556908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.557218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.557493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b0bccfea-ba26-4a84-9911-6fbd4e8f8281 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:47.566511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.567755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:47.569179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 380/743] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.569778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.569857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.569857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.569857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.570192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.579250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:47.586357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:47.587257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:47.588142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:47.588142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:47.588142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:47.588142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:47.588374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:47.588456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 364/744] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:47.588680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.589504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:47.655269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 381/745] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 19:55:47.656505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.689198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab733e-b132-49ed-8149-9e7bd3545265 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.714354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.723862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.724530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.724760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.725068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.725761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.727126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.741981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 365/746] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:47 2022] POST /identity/v3/services => generated 251 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:47.742907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.768752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59997ba8-7bc4-497a-a0f4-f7d41eb53c82 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.811618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.818960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.819874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.820150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.820421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.821158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.821463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.822856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.841690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 382/747] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.841690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.861011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3d9ac-0558-410e-85a4-5934c5572813 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.900777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.901438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.901715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.902259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.903069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.903533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.905589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:47.925648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 366/748] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:47.926486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:47.954934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d67ab2-da47-42b2-b24f-b22411553322 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:47.969919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:47.991572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:47.992257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:47.992641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:47.992917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:47.993639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:47.993931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:47.995285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.009358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 383/749] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.010186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.030659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97942b21-ad7c-4858-a7d5-17533d0239e7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.050383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.074224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.075158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.075442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.075779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.076482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.077978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.092878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 367/750] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:55:47 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:48.094696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.147456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6c491-ab8b-4f09-91f1-b073e5e778cd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.195355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.196325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.196789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.197343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.205770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.207679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.212048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e4f2e-e48c-467c-a10f-bc7468b202e4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:48.221241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.222195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:48.223106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:48.225687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 384/751] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:48.226852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.260213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.260757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.261060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.261334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.264113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.310429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.338251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.340146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.342715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b2493-0259-486f-ab4d-531bb144b275 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.378430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.379447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.379946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.380422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.388407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.390291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:48.403037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.403990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:48.404932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 385/752] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:48.405541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.480011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 368/753] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:48.481084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.517806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.528906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240286b2-d34e-49a6-80bd-4590ff4f284b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.565136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.565760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.566040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.566317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.567079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.567652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.569132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.575317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.588168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 386/754] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.589027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.601325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfce6c0-c777-480d-bffa-ac334b50967a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.640135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.641466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.641932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.642359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.643302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.643772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.645542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.665768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 369/755] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.666878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.698498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e11e63-c7bb-4d24-8670-e8b263090a88 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.734088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.735034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.735317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.735592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.746225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.747547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:48.763206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:48.764356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 387/756] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:48.766312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.768755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.785043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e72a9d-ad47-4132-b6c5-6aa87234638b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.824068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.825070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.825589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.826119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.827239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=bc49a64077c94d1194086179c518209e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:48.827753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:48.830086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:48.833616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:48.838043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 370/757] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/role_assignments?user.id=bc49a64077c94d1194086179c518209e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:48.839168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:48.913647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d29712-086e-4745-8d92-2881afc3f847 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:48.968761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:48.971188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:48.971672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:48.973227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:48.998231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.003603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.025815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794748b-adda-43f0-8c8f-56b50aeaf3e5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:49.035559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:49.036480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:49.037401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:49.037401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:49.037401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:49.037401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:49.045135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 388/758] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:49.046029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.079299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.080494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.081278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.083232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.084185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.084711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.086837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.118830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 371/759] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.121586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.173789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.196939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.256349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded1a8d-7172-431b-b216-eede332acdad None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.289705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f859b74-52fa-4729-880e-2aa3986d3b7f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.325748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.326854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.329288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.329856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.339232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.341057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:49.355648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:49.356571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:49.357827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:49.361023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 372/760] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:49.366060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.373681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.374332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.374659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.375099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.375818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.376124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.377553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.402878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 389/761] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.409245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.482100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37684f-5637-4648-a658-b2f8223d9ada None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.525337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.528336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.528987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.531034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.531353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.532689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.533041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.535305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.551144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 373/762] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:49.556865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.651805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eee75-58ef-46f7-b321-d7e1deee7656 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.674792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3d6c5-5621-4505-b78c-66b8728f10a3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.686031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.715830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.716696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.717461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.718239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.719210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=f2ee302ccbdb4fa29fefc821567048e5) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.719870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.721766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.728394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 374/763] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/role_assignments?user.id=f2ee302ccbdb4fa29fefc821567048e5&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.737787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.738571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.738926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.739277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.762372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.763866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:49.785864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:49.786848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:49.787741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 390/764] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:49.793228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:49.845920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:49.909275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892853b-9751-4739-ba0c-1bab69ac4e0b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:49.948128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:49.949147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:49.949634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:49.950109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:49.951076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:49.951658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:49.953437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd776f5-ec83-4a37-9ab0-c80b200de37c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:49.967405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 375/765] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:49.978041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.086794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5b30e-16fd-465c-ba51-693fcd0eb683 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.124074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.125196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.125828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.126304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.128572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=056e9b8fcfc84ee68a018f331e805582, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.130353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.147594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c18cb-c6bb-4c87-ac39-62be4ec290df None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.148111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c18cb-c6bb-4c87-ac39-62be4ec290df None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.148111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c18cb-c6bb-4c87-ac39-62be4ec290df None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.149690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 376/766] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:50.151130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 391/767] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:49 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:50.155099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117e5b-3d24-4721-9960-232ae4cfb479 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.155385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117e5b-3d24-4721-9960-232ae4cfb479 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.155662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c117e5b-3d24-4721-9960-232ae4cfb479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.175956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4c117e5b-3d24-4721-9960-232ae4cfb479 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:50.177235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4c117e5b-3d24-4721-9960-232ae4cfb479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:50.279499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 377/768] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:50.306776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.347266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.419408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537754-bb5a-4743-93d8-9a896523fe8d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.430557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae722c-f127-49a1-9ada-858931d70603 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.430838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae722c-f127-49a1-9ada-858931d70603 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.431118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae722c-f127-49a1-9ada-858931d70603 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.436935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 378/769] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:50.439308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.439591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.439867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.462537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:50.464025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:50.482630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.483736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.484760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.485352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.486173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.487758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.497336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:50.511389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:50.512338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:50.513223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:50.518601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 392/770] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:50.524683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.581823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 379/771] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:50.665523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.689400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.697730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57ee4e-573e-496c-86d1-b1e9bb032aeb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.770584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.771606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.772091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.772558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.773463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:50.773986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.775548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.791153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 393/772] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:50.798059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.800536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c6be-8db4-42ab-9845-e503d3460737 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:50.848047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:50.848711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:50.849010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:50.849293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] PATH_INFO: `/v3/services/network` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:50.849932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:50.851307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:50.878878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:50.879930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:50.880748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:50.884182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 380/773] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:50.885900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:50.945987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:50.984981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4b4ea-0423-4ebf-b484-d0e667e9341b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.023059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.023684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.023957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.024231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.024917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.025246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.026621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.036364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 394/774] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:51.041764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.050523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64951d-07a4-41a4-859d-124eeb48db0f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.084250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.106857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.108410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.108990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.109747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.114086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] RBAC: Adding query filter params (name=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.114755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.127992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.137947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 381/775] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.143594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.224634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20452821-68af-4ac5-9b16-2c77cb8f3939 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.272937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.273576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.273852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.274118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.274870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] RBAC: Adding query filter params (interface=public, service_id=035afda8a973440cb6cf1cfbbffc7e5d, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.275169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.276521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7be0-06be-4fce-b6e7-114a837ec2da None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.284946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 395/776] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/endpoints?service_id=035afda8a973440cb6cf1cfbbffc7e5d&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.286139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378b23-7d91-4e8f-9f5e-bbb04e5b3b2e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.286420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378b23-7d91-4e8f-9f5e-bbb04e5b3b2e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.286694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2378b23-7d91-4e8f-9f5e-bbb04e5b3b2e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.288115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 396/777] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:51.288455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.293149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.293432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.293704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.300734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d767af-2fd0-4f3d-9992-d8e154624817 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.311586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:51.312840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:51.338879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.339876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.340431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.340897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.341964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.342509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.344725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.355012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 382/778] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.356487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.382970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 397/779] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:51.383882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.405770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.474993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4531-5e6b-4df6-a1c2-bcc72ee8ada9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.492229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934a83e-9a23-491f-b7fb-f1ed82f1136c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.513176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.514227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.514702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.515348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.516525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.518238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.524693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 383/780] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.525820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.528307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.529111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.529236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.529505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.530288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] RBAC: Adding query filter params (interface=public, service_id=b3ee97b9bfde411e8e5fe01c9db13141, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.530599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.532074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.555435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 398/781] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/endpoints?service_id=b3ee97b9bfde411e8e5fe01c9db13141&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.556771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.562344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.657359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:51.669118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558495-a7cb-4103-a72a-bfda4a682dc9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.707056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.707697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.707952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.708228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.708884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.710288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:51.715976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 399/782] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:51.738482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fdada-687d-4b22-83cb-cd9d669e84de None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.778955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.779617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.779895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.780169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.792175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:51.798847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:51.799862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:51.800694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 384/783] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:51.806274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:51.952899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd56e9-b047-4160-8d85-e481759ae657 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:51.989549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:51.990101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:51.990388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:51.990660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:51.991378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:51.991681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:51.993086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c715ad-94be-4b60-b9fb-aa47da30038b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.002400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 400/784] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.008136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.096197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.119413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679f7d6-8196-4487-9753-b23f075a29e1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.140930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3050-22bf-4b0b-9476-536b6d233f65 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.141275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3050-22bf-4b0b-9476-536b6d233f65 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.141519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3050-22bf-4b0b-9476-536b6d233f65 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.142871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 401/785] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:52.149160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.149551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.149921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.165261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.166254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.166712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.167192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.172258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:52.173752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:52.186164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.188539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.190888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:52.220646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.221643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.222516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:52.222516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:52.222516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:52.222516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:52.224115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 385/786] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:52.229885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.309671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 402/787] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:52.333811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.413606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b747-237b-4697-87f3-1e540131d09f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.480241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.481292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.481800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.482277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.483189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.483696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.486076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.489258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.508313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136cf99-71ce-43be-bbb4-6635e12557ae None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.511073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 386/788] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.511946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.570763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.571329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.571612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.571888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.587574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-78c58350-3ae1-4a65-b932-246f2994dd8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:52.592548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.594059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:52.594884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:52.597168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.599341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 403/789] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:52.601287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b35bd-7ddd-4a4b-b7c3-f2806b95dd57 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.601608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b35bd-7ddd-4a4b-b7c3-f2806b95dd57 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.601876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b35bd-7ddd-4a4b-b7c3-f2806b95dd57 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.605123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 404/790] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:52.605123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b35bd-7ddd-4a4b-b7c3-f2806b95dd57 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.661467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6661e-3a56-4f2b-b124-a00b92f76c32 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.706909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b35bd-7ddd-4a4b-b7c3-f2806b95dd57 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.707294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.708170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.708560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.708923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.719413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.721328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2158c80b-c932-4474-b623-9c2e916922f9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:52.736738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:52.737737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:52.738679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:52.741291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 387/791] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:52.742811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.743330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.743818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.753687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.755173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.755457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.755786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.756514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.756876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.758361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.765928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:52.767405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:52.772316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 405/792] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:52.772316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.825434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.844222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:52.848846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 388/793] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:52.850146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.888630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95eb3-842d-4419-962b-7743d3c214d0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:52.926097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:52.927216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:52.927772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:52.928273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:52.929321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:52.929834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:52.931483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:52.950931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 406/794] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:52.952150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:52.965804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a354fcb-0096-4db6-9a00-cc823dfdfb57 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.003093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.003649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.003935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.004208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.015020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.017305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:53.033897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.034805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.035672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:53.035672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:53.035672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.035672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:53.039861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 389/795] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.040693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.065254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a15ee-d8c5-4ba1-bc67-c1977d84b87d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.079439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.090070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.106137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.106880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.107266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.107652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.119411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:53.124896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.126045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:53.127054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:53.128465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 407/796] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.129439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.158946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724c2e3-92b0-4cd5-bc0b-36e28c115f74 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.196642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.197732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.198150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.198567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.210537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.212423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:53.230234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.231222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.232100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:53.232100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:53.232100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.232100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:53.239890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 390/797] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.241256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.248064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb0b8-473e-4164-b96c-7a385454859a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.284634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.285380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.286148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.286148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.286891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.287273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.288823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.302342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 408/798] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:53.303348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.339791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.351645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.359342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239b0f5-7349-4344-ae7b-faf1bd467756 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.397071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.398120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.398685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.399141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.400032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.400514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.402198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.412345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 391/799] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:53.413440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.414890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80c66c-c82e-4d33-b795-adac805cbbc2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.454066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.454066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.454451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.454704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.455354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.455668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.457074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.474637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 409/800] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:53.474637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.530878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f60d3-a716-49d2-97b1-f51512e26a55 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.569429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.570458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.570990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.571575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.575680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.579163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ac0a0-f263-4aaa-94c3-a324e921431d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.583704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.585455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:53.599064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.599996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:53.600920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:53.603805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 392/801] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.604830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.615972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.616629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.616915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.617234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.618304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.631281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.633102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:53.648957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.649924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.650802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:53.650802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:53.650802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:53.650802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:53.652937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 410/802] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.654085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.719032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694fa1-5432-40f6-baee-a5e06ea70437 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.756067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.756879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.757274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.757616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.766000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.767367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2526df2-14b2-4470-80f9-27f3821f55dd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.767889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:53.780233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:53.782329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:53.783448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:53.788420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 393/803] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:53.789349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.810097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.811063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.811552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.812129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.813656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.814110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.816151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.834331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 411/804] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:53.837429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.849462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.867246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:53.903962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae183b-8cba-4f6b-8d03-fd226c1987bc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.941085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.941795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.942390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.942497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dd9d8-4e76-4026-aa55-80ab00561423 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:53.942781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.943543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.944016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.945601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c395f-c5cd-4081-b6b5-29f200dfc2e7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 394/805] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bafba-adcc-4958-b6cc-042ed032d89e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bafba-adcc-4958-b6cc-042ed032d89e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94bafba-adcc-4958-b6cc-042ed032d89e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 395/806] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:53.963299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bafba-adcc-4958-b6cc-042ed032d89e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:53.985424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:53.986047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:53.986766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:53.987052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:53.987811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:53.988122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:53.989594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.006266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 412/807] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:54.006266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.080032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94bafba-adcc-4958-b6cc-042ed032d89e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.083020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.112341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.118472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7984c-b259-462f-b220-44a86c34884a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.121370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.122020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.122304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.122580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.133926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.135367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-50f696cc-989f-4bd7-b294-c337d3c20c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:54.151023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:54.151921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:54.153518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:54.153518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:54.153518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:54.153518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:54.153518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 396/808] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:54.153919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.154123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.154354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.164772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.165441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.165724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.165998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.174457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.174792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:54.175893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.176017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:54.187180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:54.188453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 413/809] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:54.189443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.245018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 397/810] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:54.246116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.300453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8d2d-7149-4daa-9f97-321ba4869794 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.328572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.335328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.335993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.336277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.336554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/bc49a64077c94d1194086179c518209e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.338525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=bc49a64077c94d1194086179c518209e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.339970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732fd37-d2ac-4e45-9c94-b130bf8633a9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.348425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db416cea-15a3-43ad-8e47-39573e8f0868 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.354146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 414/811] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:54 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/bc49a64077c94d1194086179c518209e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:54.355348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b9b6-0984-4eb4-b20d-a99d93a8a93b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.355632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b9b6-0984-4eb4-b20d-a99d93a8a93b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.355907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b9b6-0984-4eb4-b20d-a99d93a8a93b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.357414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 415/812] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:54.357941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b9b6-0984-4eb4-b20d-a99d93a8a93b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.369920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.397779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.398412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.398688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.398960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.399660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:54.399992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.401407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.415480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 398/813] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:54.416197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.489193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b9b6-0984-4eb4-b20d-a99d93a8a93b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.530466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.531208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.531484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.531756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.532581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:54.532881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.534390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bbf06-88fd-4533-8b77-e174d767a18b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.535229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e490a-6874-4222-b5f1-2b8941bcc6b4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.552586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 416/814] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:54.553855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.554136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.554405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.572693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:54.573924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:54.575007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.575651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.576084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.576363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.577846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.579207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:54.589335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:54.590303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:54.591120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 399/815] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:54.591953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.627479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.634063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.661565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 417/816] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:54.661565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.714334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538f445-737e-4aad-af04-dc409dec7c0e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.751204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.752001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.752409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.752690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.759937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.760517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5de6c-c6ef-4f70-9096-2d456e2a35a5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.761491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:54.773816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:54.774751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:54.775635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 400/817] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:54.776112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:54.802040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.802759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.803061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.803318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.857824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.885058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.886618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.892352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:54.901094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610b89-8c77-4872-99e3-cdb7e4fab17d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:54.939914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:54.940533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:54.940811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:54.941514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:54.942554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:54.942830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:54.944110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:54.953804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 401/818] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:54.953804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.000357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 418/819] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:55.000357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.067431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250167a8-32d1-4a95-ad81-f2bfa0ad5311 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.082520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.109620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.110575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.110946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf492e-81e2-4e70-b09d-219647a5ea5a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.111197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.111621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] PATH_INFO: `/v3/services/network` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.112389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.113941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:55.122988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:55.124031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:55.130015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 402/820] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:55.130834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.156408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.157689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.158459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.158804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.159081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.160378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.160847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.162397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.179383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 419/821] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.179383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.246563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39050d3f-df32-4a7f-9bf7-e7092222eb1d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.284339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466be31-eb0f-4e9e-9257-0c8c6cf6c4aa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.287810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.288429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.288713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.289008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.289694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.290000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.291367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.297014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 403/822] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.298125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.324737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.325419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.325702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.326088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.333842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.337994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.339403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:55.356686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:55.357856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 420/823] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:55.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.392950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.409622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828aa38-1fab-440d-b24a-371da1928260 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.447115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.447679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.447967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.448242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.448917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] RBAC: Adding query filter params (name=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.449245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.450616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.461945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 404/824] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.462459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799988e-b8f6-4076-a7df-a2b5c74d085c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.462780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.499336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.500563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.500846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.501182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/f2ee302ccbdb4fa29fefc821567048e5/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.516791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=f2ee302ccbdb4fa29fefc821567048e5, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.518266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.531715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 421/825] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:55:55 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/f2ee302ccbdb4fa29fefc821567048e5/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:55:55.532524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.591562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.600421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd035bd7-8f31-4b97-9978-15ee7e643e36 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.651019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.652090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.653225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.653719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.654993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.658010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.666546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248f131-df16-4e0c-bb4d-b2dc29b0c920 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.696714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 405/826] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:55.699382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.713328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.725451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.726099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.726384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.726666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.727619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.727929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.730053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.747365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 422/827] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.758066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.854551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68841c1-d555-4764-b266-5c38bcacb29f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.880740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:55.889802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6258-3fd2-4364-9b2b-5f3613e1ecfd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:55.904919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.905586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.905903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.906181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.906868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] RBAC: Adding query filter params (type=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:55.907172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.908565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.914917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 406/828] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:55.922539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:55.931760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:55.932551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:55.932975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:55.933380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:55.941672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:55.944223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:55.957886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:55.958959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:55.959919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:55.961837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 423/829] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:55.974195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.016283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.064675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1bd00-f9a8-4cd1-9d0c-3a382b814c14 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.139819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.141339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.141806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.142280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.143174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.144715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.157725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9210-ac2f-43ba-972b-c7cd78834eb8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.179702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 407/830] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:55:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:55:56.226424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.227086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.227367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.227639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.228414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.228722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.230239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fac99-5342-460b-8e03-30a8f1250b4b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.249112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 424/831] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:56.256130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.393761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3c47b-5793-483c-8d3e-24d10fe88925 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.426301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.440204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.440768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.441081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.441362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.442435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.442737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.444178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b61da-ab6c-4e7a-a870-9d0e4ec71f7d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:56.450800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 408/832] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:56.510128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84617-fa4c-47f1-bee2-65369e5eb077 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.510762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84617-fa4c-47f1-bee2-65369e5eb077 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.511283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd84617-fa4c-47f1-bee2-65369e5eb077 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.513369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 425/833] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:56.526172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.526499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.526734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.551798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:56.553052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:56.643489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 409/834] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:56.671130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd84617-fa4c-47f1-bee2-65369e5eb077 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.692680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.786421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd84617-fa4c-47f1-bee2-65369e5eb077 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.823465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.824104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.824384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.824659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.835452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:56.840383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:56.841303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:56.842541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 426/835] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:56.850009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:56.956757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ee113-afd1-438d-a3c0-a4ae4c0712ea None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:56.977633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:56.993462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:56.994091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:56.994371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:56.994645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:56.995354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:56.995658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:56.997072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.006716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 410/836] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:57.012268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.087323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.118948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095a50e-adf5-4fba-8c8d-0dc1d3b20e7b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.154764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.155423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.155785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.156069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.167326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.168873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:57.184289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:57.185235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:57.186257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:57.186257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:57.186257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:57.186257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:57.186257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 427/837] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:57.193584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.302706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1829744-af93-405f-aed9-66b9372fcf9b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.339066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.339693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.339967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.340243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.340943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:57.341275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.342640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.355746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 411/838] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:57.362362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.470894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d70b6-cd0c-4af7-af45-0632ff86e35f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.497324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.507574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.508221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.508501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.508773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.517361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.518818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:57.530356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:57.531915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:57.533458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 428/839] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:57.540376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.589398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.645323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863bcf-cc6a-4e2f-9ad6-e46121ed854b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.681453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.682410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.682886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.683350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.684441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:57.684954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.686604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7be1f-e2fa-4d1c-9a01-156096b46909 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.709068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 412/840] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:57.711017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:57.822419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972d7a9-f995-40c2-8857-139c598e8853 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:57.852287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe3b6-71b4-4080-b3f3-18f3e1655b1c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.852582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe3b6-71b4-4080-b3f3-18f3e1655b1c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.852765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe3b6-71b4-4080-b3f3-18f3e1655b1c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.854396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 413/841] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:55:57.859807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.860092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.860284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:57.860400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.860923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:57.861248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:57.861526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:57.878863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:57.880062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:57.919858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:57.938586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:57.940234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:57.981010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 414/842] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:55:58.002738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.036070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.080896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 429/843] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:55:58.089158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.121491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390be33-d22a-40b7-b13b-468faf65fc2e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.161095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.161745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.162038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.162313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.173956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:55:58.176577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0c221fa1-5303-4a30-9602-a984c002c8bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:55:58.178989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:58.180282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:58.181176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 415/844] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:55:58.181877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a228ecc-8cd4-4209-a12d-b3b3cc12ae87 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.182836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a228ecc-8cd4-4209-a12d-b3b3cc12ae87 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.182836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a228ecc-8cd4-4209-a12d-b3b3cc12ae87 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.184068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 416/845] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:58.187487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a228ecc-8cd4-4209-a12d-b3b3cc12ae87 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.199813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab71fb-012a-4100-ab43-937bfd4cb8c6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.237529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.238172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.238443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.238713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.249621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.251270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-6570db42-3295-4a6f-bdf9-7bf6b3c915a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:58.267830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:58.268770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:58.269685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:58.269685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:58.269685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:58.269685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:58.273271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 430/846] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:58.275082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.275368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.275649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.287500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.294486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:58.295783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:58.299562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a228ecc-8cd4-4209-a12d-b3b3cc12ae87 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.340150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.340992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.341384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.341494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.342211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.342521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.344175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.353001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 417/847] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.353774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.371647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 431/848] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:58.372278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.444350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.472217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8b332-82d1-4588-b90c-1c8f97dfa18c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.481920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb363a-25c3-4a0e-b677-c6e290b4f09f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.509021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.509647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.509931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.510206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.510892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.511197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.512555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.518364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.519005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.519607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.519607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.524537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 418/849] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.525210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.535091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.536549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.541546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:55:58.552703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:58.553653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:58.554523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:55:58.554523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:55:58.554523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:55:58.554523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:58.554523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 432/850] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:58.554819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.637074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511663f-49f9-41af-9ee9-40a6aadd8b14 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.674942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.675614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a8bc0-c324-4150-b9df-b3e7e837638b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.675701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.675934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.676121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.676754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.678157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:58.687123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:58.688544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:58.689421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 419/851] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:58.690214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.698057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.719728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.720486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.720778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.721069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.727870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.729346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:55:58.741969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:58.742926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:58.743800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 433/852] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:58.744281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.777509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.802587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f198b-9740-4c04-89b8-742b731c0ee7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.835918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.836537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.836816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.837122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.837814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.838121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.839482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66385e15-3ab2-4ff3-8e4a-56272b3b8a8c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.845942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad8b4-22fc-4569-a7f4-512f8498b695 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.851968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 420/853] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.853002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee33035-03f1-4783-8808-3e361a1c2e05 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.853294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee33035-03f1-4783-8808-3e361a1c2e05 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.853572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee33035-03f1-4783-8808-3e361a1c2e05 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.854840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 421/854] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:55:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:55:58.855476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee33035-03f1-4783-8808-3e361a1c2e05 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.884447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961be53-2870-4822-8004-111688905e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:58.885103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961be53-2870-4822-8004-111688905e14 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:58.885387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961be53-2870-4822-8004-111688905e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:58.885661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961be53-2870-4822-8004-111688905e14 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:58.886342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961be53-2870-4822-8004-111688905e14 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:58.886644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961be53-2870-4822-8004-111688905e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:58.888070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961be53-2870-4822-8004-111688905e14 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:58.892905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 434/855] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:58.893713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961be53-2870-4822-8004-111688905e14 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:58.941128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:58.966521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee33035-03f1-4783-8808-3e361a1c2e05 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:58.998127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961be53-2870-4822-8004-111688905e14 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.001318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.001941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.002225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.002498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.003244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.003550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.004983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf3d3a-554e-427e-9bb4-f27cb9618f61 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.007728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.017768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 422/856] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:59.019040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.019333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.019611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.037494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:55:59.037683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.038428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.038691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:55:59.038785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.038969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.047130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.048564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:55:59.059578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:59.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 435/857] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:55:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:59.061540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.114731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 423/858] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:55:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:55:59.115400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.165699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ea676-7de6-47bc-9879-e26e2d9fbd61 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.167117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.201377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.202001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.202302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.202868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.203998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.204312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.205723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.211379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 436/859] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:59.212304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.224109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186b7ef-cb85-45c3-bad7-ba6feae99f0b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.255936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.269882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.270570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.270937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.271306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.272178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=bc49a64077c94d1194086179c518209e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.272566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.274310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.293450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 424/860] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/role_assignments?user.id=bc49a64077c94d1194086179c518209e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:59.293450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.372257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860dd86-8ccd-47b1-9bdd-39ef296a59b6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.415405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.416097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.416324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.416604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.418270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.418589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.420096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.431809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b3d42-afb2-43fc-bf9f-c98b4b19ddb3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.441559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 437/861] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:59.447711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.472924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.473653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.473873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.474155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.474791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.476170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.478659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:55:59.486253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:55:59.487384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:55:59.488303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 425/862] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:55:59.496448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.578489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.583333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffe797-fc6f-446d-a91b-bbe3b0af9054 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.621915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.622542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.622823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.623099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.640116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db0b2-e9b5-4e08-b062-a1298aad0250 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.686399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.688838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.689303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.689986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.690837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.691556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.694085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.699634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.700817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 426/863] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:55:59.701424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.706810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.728079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.821738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 438/864] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:55:59.827270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:55:59.841520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a69a-7a1c-4832-b80d-692b428a2d7a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.860830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:55:59.879680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:55:59.880326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:55:59.880608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:55:59.880884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:55:59.881620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:55:59.881927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:55:59.883294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:55:59.888542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 427/865] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:55:59.960416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6dfb-b733-4a78-ad70-9e5d29a79fff None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:55:59.989439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.002715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.003661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.004103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.004528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.017141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.018920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:00.035678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:00.036855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:00.037992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:00.037992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:00.037992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:00.037992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:00.039946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 439/866] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:00.048266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.175326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc49ac-840c-494d-83e6-d1aedb36e746 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.212456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.213605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.214168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.214658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.215602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:00.216103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.217695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.231064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 428/867] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:00.237170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.340043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3347a-472b-47ea-8b97-0fa7ce9e5ab1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.374911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.375558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.375841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.376117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.383171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.384634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:00.397023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:00.397982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:00.399080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 440/868] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:00.404218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.452274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.514375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197c2f-2895-44ff-b5d0-dd5db9a9ab90 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.551645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.552623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.553181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.553677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.554644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:00.555158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.556797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e3410-e98e-4132-9c56-12062d9489f3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.572753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 429/869] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:00.584990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:00.617825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:00.690203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd022f9c-12a4-41cc-a5cd-b4fc0ab95057 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:00.725830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:00.726452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:00.726735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:00.727013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:00.727807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=f2ee302ccbdb4fa29fefc821567048e5) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:00.728121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:00.729604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc4b2a-ee2a-465c-a249-1022803d98d8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:00.736233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 441/870] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:00 2022] GET /identity/v3/role_assignments?user.id=f2ee302ccbdb4fa29fefc821567048e5&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:01.323761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bf9b-6da2-47dd-ab81-625dca97e979 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.324347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bf9b-6da2-47dd-ab81-625dca97e979 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.324347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9bf9b-6da2-47dd-ab81-625dca97e979 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.325958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 430/871] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:01.331925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461329-0ae9-4cc0-a4d5-7fc33db4c2db None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.332320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461329-0ae9-4cc0-a4d5-7fc33db4c2db None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.332547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461329-0ae9-4cc0-a4d5-7fc33db4c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.351445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-23461329-0ae9-4cc0-a4d5-7fc33db4c2db None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:01.352629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-23461329-0ae9-4cc0-a4d5-7fc33db4c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:01.427497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 442/872] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:01.450170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9bf9b-6da2-47dd-ab81-625dca97e979 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.523566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b379c9-09a4-4d78-8ad7-32cbc4100574 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.523897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b379c9-09a4-4d78-8ad7-32cbc4100574 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.524136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b379c9-09a4-4d78-8ad7-32cbc4100574 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.525816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 443/873] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:01.531788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.532370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.532651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.556664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:01.558552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:01.568397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9bf9b-6da2-47dd-ab81-625dca97e979 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.570196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.594718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:01.609872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.610746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.611095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.611436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.612211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.613892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.632114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 431/874] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:01 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:01.654078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 444/875] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:01.678409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.793825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbcba8-11d9-4eec-b201-463e6d54c67e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:01.830543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:01.831188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:01.831470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:01.831763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:01.832375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:01.833772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:01.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:01.844708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:01.845656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:01.849072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 432/876] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:01.854049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:01.965537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927fbc2-fdaa-4552-abaf-8e5195909362 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:02.002005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:02.002633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.002916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.003190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.003874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:02.004179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:02.005594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34460209-e4c4-438a-9c61-0bfe72814752 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:02.010708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 445/877] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:02.017460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:02.047666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:02.103895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48726-be30-4863-ba43-cc11297b6b7f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.103895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48726-be30-4863-ba43-cc11297b6b7f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.104165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48726-be30-4863-ba43-cc11297b6b7f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.105893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 446/878] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:02.111682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ca20-788a-45ec-9824-0ac9050e34a1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.111972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ca20-788a-45ec-9824-0ac9050e34a1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.112305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663ca20-788a-45ec-9824-0ac9050e34a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.130648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5663ca20-788a-45ec-9824-0ac9050e34a1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:02.131197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91efbc9-d33e-45b2-b78c-7e70b7c3176e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:02.131839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5663ca20-788a-45ec-9824-0ac9050e34a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:02.146751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:02.172920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:02.173926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.174411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.174907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.175892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:02.176412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:02.178102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:02.189113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 433/879] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:02.205779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 447/880] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:02.233869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:02.342562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:02.355404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a92f8-b201-41e4-934f-79e7657df4cf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:02.394624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:02.395252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.395533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.395810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.396509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:02.397926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faba3aa-f69d-4b49-b840-ac5a3bb625d1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:02.421411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 434/881] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:02 2022] POST /identity/v3/services => generated 269 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:02.906429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277e62f-b4f7-4ab2-93e4-6d023b6ded33 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.907166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277e62f-b4f7-4ab2-93e4-6d023b6ded33 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.907166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277e62f-b4f7-4ab2-93e4-6d023b6ded33 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.908684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 448/882] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:02.913821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:02.914116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:02.914402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:02.933784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:02.934993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:03.034950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 435/883] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:03.106080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277e62f-b4f7-4ab2-93e4-6d023b6ded33 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.267575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277e62f-b4f7-4ab2-93e4-6d023b6ded33 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.337277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:03.337930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.338213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.338491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.369930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:03.375038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:03.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:03.377064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:03.381273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 449/884] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:03.382321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.491002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.515867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c934af-423b-4d70-87bb-5f21e17feb37 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.550550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:03.551098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.551390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.551666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.552451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:03.552762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:03.554285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:03.563925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 436/885] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:03.569748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.583553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.678685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b712ae-c0dd-4263-9513-ff9b2f6b7e67 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.715078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:03.715724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.716006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.716286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.726702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:03.728154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-7513361b-e22a-4f5e-a46f-4dec2d6846eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:03.742998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:03.745118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:03.746114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:03.746114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:03.746114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:03.746114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:03.746114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 450/886] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:03.751250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.777756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be149ef-450c-4425-9c2e-d8256719ea28 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.778031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be149ef-450c-4425-9c2e-d8256719ea28 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.778549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be149ef-450c-4425-9c2e-d8256719ea28 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.780363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 451/887] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:03.786185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.786655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.787151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.809209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:03.810605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:03.864724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392f41e-d702-475a-b46b-d9231ffda338 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:03.889649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.904507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:03.906073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:03.906586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:03.907120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:03.908041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:03.908557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:03.909227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 452/888] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:03.910726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:03.918924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:03.930151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 437/889] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:03.931168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:03.935382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.040782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f49ea-68f1-4c08-927b-4f94f7d84d91 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.062556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1efa9-dc03-48cf-893e-4bb3a07f0d5c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.084889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.085875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.086360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.086857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.093266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.095001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:04.105936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:04.106923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:04.107748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:04.110293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 438/890] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:04.113110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.114005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.114251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.114778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.115221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.125578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.127802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:04.144549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:04.145577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:04.146509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:04.149433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 453/891] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:56:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:04.156520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.188086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.229082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.235108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96c1b0-5372-4cbc-ab50-e45b0d799236 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.273297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.274299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.274815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.275294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.276177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.276695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.278303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.289174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 439/892] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:04.290310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.291329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3786f1-1caf-48b1-9037-e6613cc4bcdd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.334753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.335739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.336227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.336701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.337749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.338315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.339953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.364321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 454/893] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:04.365557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.416022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d2f53-a074-448d-bd59-0b01615ba294 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.442909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.453319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.453948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.454223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.454493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.455165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.455463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.456819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283d0e6-9f89-484e-bf61-1230beedf9b5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.466540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 440/894] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:04.466540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0ea5-de99-4c8a-b0b1-a0fc393ebc21 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.466540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0ea5-de99-4c8a-b0b1-a0fc393ebc21 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.466540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f0ea5-de99-4c8a-b0b1-a0fc393ebc21 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.466540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 441/895] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:04.468416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f0ea5-de99-4c8a-b0b1-a0fc393ebc21 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.477597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86187934-1318-442f-8ab0-3478e5af5a2e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.489013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.515639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.516269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.516572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.516830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.581694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f0ea5-de99-4c8a-b0b1-a0fc393ebc21 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.586551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.587961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842fdf5-76fa-481d-889d-95c3cd9915ee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.619390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.620019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.620305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.620585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.621384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] RBAC: Adding query filter params (interface=public, service_id=dbf6893c813e41808d425193046a1915, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:04.621698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.623063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34689d-2214-4d4b-8bdb-620522a8f2eb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.635863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 442/896] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/endpoints?service_id=dbf6893c813e41808d425193046a1915&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:04.635863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.635863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.635863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.651270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:04.652468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:04.689803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 455/897] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:04.691151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbccf4-f1e1-4e3a-b033-c8ae126d000f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.691444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbccf4-f1e1-4e3a-b033-c8ae126d000f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.691721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbccf4-f1e1-4e3a-b033-c8ae126d000f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.693411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 456/898] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:04.694027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbccf4-f1e1-4e3a-b033-c8ae126d000f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.706271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.706979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.726268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 443/899] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:04.727286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.796873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbccf4-f1e1-4e3a-b033-c8ae126d000f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.831608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.832215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.832502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.832772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.833446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.834811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75135a19-2b40-44aa-b556-aa292e8861af None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.836105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa0449-de83-4766-8a07-30fb3136f476 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:04.840461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 457/900] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:04.841563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.841848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.842119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.861798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:04.863041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:04.876526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:04.877221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:04.877507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:04.877778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:04.889368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:04.891437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:04.908839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:04.909977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:04.910868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:04.910868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:04.910868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:04.910868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:04.910868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 444/901] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:04.911154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.948886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:04.949762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 458/902] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:04.950623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:04.955440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.039785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27271a0-6118-4ec4-ad04-3f0134f916ae None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.082599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc0434-8a38-4677-b087-c17e550be825 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.083422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.084042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.084323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.084593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.085310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.086669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.106862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 445/903] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:04 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:05.107627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.132166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.132905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.133295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.133644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.134437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.134818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.136471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.154567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 459/904] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:05.156365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.228759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.266998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.275329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed615548-8492-4ecc-8860-8e901dfce652 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.286427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae877a-f22b-414a-8ec0-ba63a6bd02fe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.321737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.322717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.323206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.323558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.324202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.325611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.328807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.329833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.330216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.330505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:05.336284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:05.337276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:05.338114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 446/905] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:05.340402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.342400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.345218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:05.356786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:05.357992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:05.359280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 460/906] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:05.363852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.470453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a4868-5726-43b0-bcce-3cbde5126e0f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.476696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92969de2-93be-4635-9c6e-22fc6a2c4f9a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.497981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.509937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.510611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.510839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.511115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.511975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.512284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.513780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.515109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.515948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.516361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.516742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.517654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.518066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.519200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 447/907] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:05.521383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.524935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.545046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 461/908] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:05.547820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.564463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.670135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4751a9-be35-4d39-b55d-47bcf36337b3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.674859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c13282-e67b-4f14-b831-25157a3d8fcd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.716805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.717455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.717735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.718596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.719281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.720150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.721559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.724436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.725587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.726092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.726576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.727570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] RBAC: Adding query filter params (role.id=5bd19a2337764593906955917f1ac831, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=f2ee302ccbdb4fa29fefc821567048e5) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.728103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.729788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.731827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 448/909] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:05.732702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.738722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 462/910] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/role_assignments?user.id=f2ee302ccbdb4fa29fefc821567048e5&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=5bd19a2337764593906955917f1ac831 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:05.839018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:05.854765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60c3b0-8219-4f1d-8923-904b3d8e8793 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:05.894795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:05.895782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:05.896291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:05.896769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:05.897808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] RBAC: Adding query filter params (interface=public, service_id=03ffa8dfc8fa48d4905909b2dbb63a80, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:05.898347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:05.899975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bac74-9fb2-4a4b-aee3-dead88b7f09e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:05.906445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 449/911] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/endpoints?service_id=03ffa8dfc8fa48d4905909b2dbb63a80&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:05.912243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:05.967620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:06.029109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbaa98-f41a-47bf-9dcb-61a9e7a57208 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:06.066612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:06.067872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.068415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.068893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.069833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:06.071406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ae7c-d090-4f98-ab5f-835a9f0b4dec None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:06.078530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 463/912] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:05 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:06.905168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69543-5473-411d-88fa-88db29896354 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.906462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69543-5473-411d-88fa-88db29896354 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.906462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69543-5473-411d-88fa-88db29896354 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.908065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 450/913] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:06.913155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:06.913439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:06.913713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:06.932435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:06.933645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:07.005675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 464/914] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:07.027016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb69543-5473-411d-88fa-88db29896354 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.143252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb69543-5473-411d-88fa-88db29896354 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.179710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.180695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.181218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.181712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.182561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.184136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-cf0c65a1-9e5b-44d0-85a9-8b99ae0d6fca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:07.194171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:07.195253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 19:56:07.196172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.199165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 451/915] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:07.206051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.243907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163408-368a-4fe2-a57f-385bf12cd235 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.244247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163408-368a-4fe2-a57f-385bf12cd235 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.245101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163408-368a-4fe2-a57f-385bf12cd235 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.247731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 452/916] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:07.248262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.252575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.253245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.253533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.275748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:07.277408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:07.289912 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.327271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9923c6-bf8b-4567-802e-753b0d6e8d87 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.363932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 453/917] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:07.368544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.369449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.369928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.370396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.371278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.371849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.373531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.379699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 465/918] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:07.385406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.385546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.489902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c4e52-4d40-4790-b566-3248b59aa2d2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.491734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.492669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b319-4dd5-4074-a5d4-7a7d53096d3a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.529297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.529534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.529995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.530278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.530773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.531117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.531531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.531898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.532390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.533699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.534214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.535792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:07.546154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:07.547268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:07.548119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 454/920] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:07.548905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 466/920] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:07.549003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.550097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.566426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.679845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfb3-c0b6-49a9-8613-d329e9dd4027 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.685510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762521a-8742-43da-bfd1-431b468de542 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.720506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.721804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.722687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.723440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.724581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.725517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.727640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.730147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.730752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.731039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.731316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.732031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.733964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf060-77a7-483c-9398-e823c70da871 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.735708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 467/921] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:07.743938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.767954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 455/922] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:07.781310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:07.869316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4910b-0ad8-4896-b5c1-5d57e5525e60 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:07.871388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a56f0-6e5a-4f8f-ac30-6468a99fd41d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.871680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a56f0-6e5a-4f8f-ac30-6468a99fd41d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.871962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a56f0-6e5a-4f8f-ac30-6468a99fd41d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.873669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 456/923] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:07.879241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.879694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.879987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.900354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:07.901624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:07.927117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:07.927745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:07.928025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:07.928300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:07.929067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:07.929374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:07.930757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:07.936307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 468/924] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:07.941985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:07.984321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.000267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 457/925] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:08.023115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.057264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.070810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b485-8e90-4529-a8db-7ae2f9eb95fc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.116238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.117197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.117679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.118123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.119451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] RBAC: Adding query filter params (interface=public, service_id=163dffc0ec7847deb9ca2e8f0fc9c522, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.120193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.121941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177495-7617-4410-9eb3-c39941aed71b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.130834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 469/926] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:07 2022] GET /identity/v3/endpoints?service_id=163dffc0ec7847deb9ca2e8f0fc9c522&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:08.136485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27e170-0cbe-4b6c-87f0-aba8dddcfaab None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.136777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7124ea-9115-4865-89d7-53bd48ac5bd1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.137086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7124ea-9115-4865-89d7-53bd48ac5bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.137361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7124ea-9115-4865-89d7-53bd48ac5bd1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.138792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 470/927] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:08.139347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7124ea-9115-4865-89d7-53bd48ac5bd1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.179883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.180625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.181009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.181354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.193958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-96fe2a33-d1db-4fb3-a900-2bb5ed6d0c89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:08.199148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.200130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:08.201486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 458/928] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:08.202264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.202264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.202264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.220533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:08.221756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:08.256011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7124ea-9115-4865-89d7-53bd48ac5bd1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.281241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.292825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.293482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.293763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.294042 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.294687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.297054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.304513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 459/929] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:08.304513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.304903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 471/930] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:08.324207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.346222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.424267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a53ce1-fc3e-4a39-b91f-bb6e647c5977 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.449090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355c854-985c-465d-b30d-0052c1ea6650 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.462954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.463586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.463920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.464202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.464931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.465284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.466668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.481290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 460/931] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.484051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.501705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.502353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.502630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.502900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.503525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.504897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.517419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.518412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:08.519245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 472/932] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:08.524760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.559870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.614949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.627841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698b32-29b1-4ee1-8840-32ae45033f50 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.640531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e9ab-e2d0-4f48-a73e-b2aa18eb3d2a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.677293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.678150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.678589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.678760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.679173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.679531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.679909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.680283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.680435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.680865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.682393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.692976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.693274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 473/933] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.694312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.694484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:08.714503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:08.715429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:08.716339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:08.716339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:08.716339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:08.716339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:08.721640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 461/934] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:08.722578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.820667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa06cf4-0394-4671-ace4-e52c4bb5205b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.836550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:08.840612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4c613-6770-44d9-b905-8cd0b60f35f5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:08.868574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.869319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.870064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.870419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.871438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.871839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.873979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.879011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:08.880008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:08.880996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 474/935] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.885040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:08.885332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:08.886048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:08.886336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.886468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:08.887815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:08.906650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 462/936] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:08.910016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:08.912290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.023814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9b9f-b7e8-48dd-b3e5-e5cb037105f4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.038909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4651167-9934-4683-a548-ef5efa17ac04 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.076478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.077490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.077884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.078362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.078938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.080124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.080540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.080981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.081846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.083509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.086836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.091983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:09.109446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:09.110516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:09.111516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:09.115449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 463/937] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:56:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:09.116273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.123537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 475/938] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:56:09.141807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.230312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3f5d4-ac65-49f4-b9c8-3de6d6dea737 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.275136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.275765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.276048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.276336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.277150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:09.277460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.278906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.298567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 464/939] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:09.304235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.379383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.420652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9ebf7-4acd-4c3b-83b6-4359c1e234d6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.457251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.457904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.458185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.458454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.474135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.475982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:09.492326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:09.493282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:09.494156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:09.494156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:09.494156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:09.494156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:09.496371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 476/940] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:09.503233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.582458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.610107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10eef2-f8d6-4684-bd1b-b3b1d37deb1a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.649801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.650422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.650702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.650978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.651665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:09.651968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.653367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.665940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 465/941] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:09.672243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.777997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41ad43-141e-46ac-8e41-e312001f8892 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.796106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:09.815804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:09.816804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.817576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.818050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.825207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:09.827032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c8a28926-54b8-4d3c-91ee-e08fc6bdd931 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:09.839829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:09.840758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:09.841688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:09.844314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 477/942] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:09.850464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:09.892615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7ebc-a47e-4171-8304-62b9eba1becf None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.892948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7ebc-a47e-4171-8304-62b9eba1becf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.893209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b7ebc-a47e-4171-8304-62b9eba1becf None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.894839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 478/943] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:09.899926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584119-976f-42f1-949c-7b028939da5c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:09.900184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584119-976f-42f1-949c-7b028939da5c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:09.900423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb584119-976f-42f1-949c-7b028939da5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:09.919629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cb584119-976f-42f1-949c-7b028939da5c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:09.920823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cb584119-976f-42f1-949c-7b028939da5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:09.973475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581660bb-db03-4d19-bce1-f71853b7772f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:09.999595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 479/944] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:10.016777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.017491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.017774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.018052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.018817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:10.019126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.020691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.022399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.027185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584119-976f-42f1-949c-7b028939da5c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.042437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 466/945] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:10.043667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.148799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb584119-976f-42f1-949c-7b028939da5c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.160171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.160540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088b8ab-a0e0-44be-9335-51187c7f8bed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.188308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.189306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.189791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.190285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.191148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.193114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.196214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING keystone.server.flask.common [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 19:56:10.199011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.199687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.199972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.200252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/f2ee302ccbdb4fa29fefc821567048e5/roles/5bd19a2337764593906955917f1ac831` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.202919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=f2ee302ccbdb4fa29fefc821567048e5, role_id=5bd19a2337764593906955917f1ac831)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.204354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.216555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.sql.core [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:56:10.216555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 19:56:10.216555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [parameters: {'id': '9327847891224d39922d6099a95a3ea1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 19:56:10.216555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80136) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:56:10.221000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 467/946] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:10 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/f2ee302ccbdb4fa29fefc821567048e5/roles/5bd19a2337764593906955917f1ac831 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8a3d0c0c-308e-4f12-8aa0-bebb69a59b0e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9327847891224d39922d6099a95a3ea1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9327847891224d39922d6099a95a3ea1. Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:56:10.241469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 19:56:10.242408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application yield resource Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise self.value Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application session.commit() Jul 27 19:56:10.243364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:56:10.244288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:56:10.245223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:56:10.246015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [parameters: {'id': '9327847891224d39922d6099a95a3ea1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.246876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:10.247657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9327847891224d39922d6099a95a3ea1. Jul 27 19:56:10.250756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.254017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 480/947] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Jul 27 19:56:10 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:10.261487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.327965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.387073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47f59c-c7fd-44a1-a03a-5992000d7b44 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.415995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82d96d-c9c3-489b-b9ff-28d57e2f7b16 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.416201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82d96d-c9c3-489b-b9ff-28d57e2f7b16 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.416466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82d96d-c9c3-489b-b9ff-28d57e2f7b16 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.420778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 481/948] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:10.427041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.427769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.428209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.444090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.445049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.445162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.445379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.453223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.453766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:10.454830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.455306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:10.468277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:10.469512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:10.470459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 468/949] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:10.475686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.553072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 482/950] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:10.576264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.592181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de826e0d-d45f-4145-82db-0a0b8ce1cd10 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.607568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.635099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.636184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.636650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.637147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.638051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:10.638611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.641033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.660157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 469/951] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:10.688138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:10.696023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6d4bc-f520-42b1-ad6e-6f27b2cd445f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.737761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.738427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.738712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.738989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.739632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.741203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:10.751776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:10.752831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:10.753937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 483/952] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:10.761821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:10.877299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6530-50c4-47ff-9bc4-15d7fc09dc40 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:10.913672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:10.914640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:10.915128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:10.915636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:10.916529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:10.917099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:10.918674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:10.925305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 470/953] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:10.930657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.036885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe7b8a-8e5e-42ed-8a63-aa69c1cb502e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.074622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.074868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.075051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.081114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.081114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.081114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.081114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b681311-c601-49b1-8fb6-1261a82ebd13 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.083908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 484/954] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:11.090337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.133107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.198791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56672f3-1528-4c6d-9c19-fd06e7f31702 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.235624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.236658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.237166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.237640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.238550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.240105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a71d0-89fe-4d41-99a0-1eb9c0d7672a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.262922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 471/955] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:11.331483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0caa42-7a75-4b6b-9fff-b3a93bb1084e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.331985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0caa42-7a75-4b6b-9fff-b3a93bb1084e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.331985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0caa42-7a75-4b6b-9fff-b3a93bb1084e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.333777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 485/956] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:11.339290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.339598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.339852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.365642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:11.367260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:11.452227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 472/957] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:11.473789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0caa42-7a75-4b6b-9fff-b3a93bb1084e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.500819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.583037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0caa42-7a75-4b6b-9fff-b3a93bb1084e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.617307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.617951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.618233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.618509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.619135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.620505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:11.630115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:11.631216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:11.632735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 486/958] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:11.637258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.659409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.741348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722eca6-5978-4ef5-bf89-e778f0543a3a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.778922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.779917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.781022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.781514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.782486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.782989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.784568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fb24a-68f6-45d2-97a3-9aef472c1cc8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.790368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 473/959] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:11.798248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:11.887242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:11.909097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c6f0f-1c8f-4ab5-8d4c-cc26db36f585 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:11.944844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:11.945516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:11.945800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:11.946079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:11.946766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:11.947071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:11.948447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e432-f070-48d2-b94b-592f5f2057cb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:11.954287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 487/960] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:12.455553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810521e-224e-4146-a623-13340f6577a8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.456409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810521e-224e-4146-a623-13340f6577a8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.456837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810521e-224e-4146-a623-13340f6577a8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.458626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 474/961] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:12.464259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.465172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.465658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.485023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:12.486574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:12.561067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 488/962] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:12.580577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810521e-224e-4146-a623-13340f6577a8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.672565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.691950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810521e-224e-4146-a623-13340f6577a8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.728352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.729334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.729842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.730339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.741855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:12.747185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:12.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:12.749517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:12.756984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 475/963] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:12.758582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.853171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:12.873026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa3886-61b3-42a0-9c86-73bc8dfeebef None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:12.910218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:12.910848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.911132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.911410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.912137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:12.912442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:12.913848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c4d30-89ea-4931-8ecc-20966198a3d9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:12.922813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 489/964] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:12.928594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:12.954308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83e24-b3c2-4a9b-9ad3-a635bda702cc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.954609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83e24-b3c2-4a9b-9ad3-a635bda702cc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.954867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b83e24-b3c2-4a9b-9ad3-a635bda702cc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.956475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 490/965] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:12.961374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:12.961661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:12.961951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:12.984344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:12.986720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:13.052811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0282f06-8526-4d9c-b232-dd06fd94ff3c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.077854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 491/966] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:13.109495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.112316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.112995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.113396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.113683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.129126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.130629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.141122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:13.157070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:13.158058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:13.158937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:13.158937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:13.158937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:13.158937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:13.166708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 476/967] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:13.168097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.198579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.232470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfcedc-a7f3-452a-a08a-c1dd21c76075 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.274043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.275313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.275921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.276453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.277317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.279794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.299278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfae46-f945-4c53-9947-d9b34481de94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.302555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.sql.core [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:56:13.302555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 19:56:13.302555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [parameters: {'user_id': '6c3a5df4235b4faf9700b792fa92925c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 19:56:13.302555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80136) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jul 27 19:56:13.311262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:56:13.313117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application yield resource Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise self.value Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application session.commit() Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 19:56:13.314071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.flush() Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application compat.raise_( Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 19:56:13.314941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 19:56:13.323201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 19:56:13.325758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c3a5df4235b4faf9700b792fa92925c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:13.328680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 19:56:13.330664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 492/968] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Jul 27 19:56:13 2022] POST /identity/v3/users => generated 160 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:13.333349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.355917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.356630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.357015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.357370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.358134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:13.359189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.360519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58293d18-674f-43bb-8c79-474bb4adddd8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.372817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 477/969] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:13.374127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e239-f343-4c6a-a980-9cc80336b1ed None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.374418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e239-f343-4c6a-a980-9cc80336b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.374693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5e239-f343-4c6a-a980-9cc80336b1ed None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.376077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 478/970] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:13.381868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e239-f343-4c6a-a980-9cc80336b1ed None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.444258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.449456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb31fd9-963a-405f-ac0b-3209b0a99784 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.488441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.489104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.489389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.489668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.491974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.497829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.499917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.503584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5e239-f343-4c6a-a980-9cc80336b1ed None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-92721523-3f1d-40b4-a1a4-ab6df38cf924 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:13.512497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:13.513671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:13.514706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 493/971] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:13.515690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92beef-1920-49ac-9716-342177f6c899 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.515690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92beef-1920-49ac-9716-342177f6c899 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.515938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92beef-1920-49ac-9716-342177f6c899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.534448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-de92beef-1920-49ac-9716-342177f6c899 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:13.535929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-de92beef-1920-49ac-9716-342177f6c899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:13.551175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.552374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.552833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.553403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.563423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.564624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:13.576806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:13.578058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 19:56:13.579233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:13.585613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 479/972] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:13.585613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.615818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 494/973] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:13.616921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92beef-1920-49ac-9716-342177f6c899 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.705135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cad27-9021-40a1-8797-f26d17fda1f6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.722729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.731697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92beef-1920-49ac-9716-342177f6c899 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.743480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.744183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.744463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.744737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.745533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:13.745841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.747571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.756947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:13.768210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d1da-d181-4f64-8110-743468462946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.768976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d1da-d181-4f64-8110-743468462946 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.769382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d1da-d181-4f64-8110-743468462946 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.769758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417d1da-d181-4f64-8110-743468462946 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.770657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417d1da-d181-4f64-8110-743468462946 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:13.771066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417d1da-d181-4f64-8110-743468462946 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.772593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417d1da-d181-4f64-8110-743468462946 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.797073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 480/974] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:13.797073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.809449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 495/975] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:13.812101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d1da-d181-4f64-8110-743468462946 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:13.926464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745785bb-171c-4aa9-932b-e000e075df5e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.948708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417d1da-d181-4f64-8110-743468462946 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:13.976587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.977659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.978162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.978664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.979406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:13.980865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:13.990735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:13.991800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:13.992698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 481/976] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:13.994980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:13.995604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:13.995882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:13.996154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:13.998361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.055199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.065422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.087135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.088606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d627b-522d-425b-a0b8-022737ced866 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.125196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae7654-1466-4cc0-9863-fc5ade446130 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.159658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.160272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.160555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.160830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.161563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:14.161864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.163225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69362e-7e10-4fa2-9283-bee006d6e4fd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.167836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 482/977] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:14.174166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e61f-b98f-46e3-b576-0f9fcba5a4c2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.174458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e61f-b98f-46e3-b576-0f9fcba5a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.174742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192e61f-b98f-46e3-b576-0f9fcba5a4c2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.176006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 483/978] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:14.176700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192e61f-b98f-46e3-b576-0f9fcba5a4c2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.190302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 496/979] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:14.190302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.190302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.190599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.207798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:14.209004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:14.274826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 497/980] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:14.275645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.288364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192e61f-b98f-46e3-b576-0f9fcba5a4c2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.289708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.291050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.324870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.325438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.325717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.325988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.326661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:14.327181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.328691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.334023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 484/981] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:14.334858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.381821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939099b1-a5d9-414b-bda1-5a97575d7d9d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.415622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.416261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.416539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.416811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.427076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.428507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.432193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42419a-7edd-4d43-885a-6e7149f58cdc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:14.443330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:14.444284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:14.445237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:14.445237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:14.445237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:14.445237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:14.445237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 498/982] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:14.445519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.470308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.470936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.471211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.471486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.472175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.473570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.488024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 485/983] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:14 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:14.488828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.516378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.532392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.565292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cc68d-0940-43ae-acf0-b3c0216b91e2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.613659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.614350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.614653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.614922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.615707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] RBAC: Adding query filter params (interface=admin, service_id=163dffc0ec7847deb9ca2e8f0fc9c522, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:14.616020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.617513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.618761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d044bd-ef22-4f68-b59c-762d0e603105 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.628441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 499/984] 10.222.0.65 () {56 vars in 1249 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/endpoints?service_id=163dffc0ec7847deb9ca2e8f0fc9c522&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:14.629609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.662470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.662784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.663063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.663335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.664023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:14.664328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.665718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.680164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 486/985] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:14.687919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:14.741213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ecedbd-4223-4d3b-b1de-2983a7cba247 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.765984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.781517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.782147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.782427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.782705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.783356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.784727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.791081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:14.791529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 500/986] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 19:56:14.824370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab816ae-12e2-45f7-8dce-288813f2c2b8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:14.864853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:14.865478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:14.865763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:14.866041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:14.873703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:14.875224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:14.887928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:14.888870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:14.890116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:14.890937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 487/987] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:14.895411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.004810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fc2b7-bbc0-4fa2-835b-d626ee173d90 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.039526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.040416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.040859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.041294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.042185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:15.042629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.044191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8f251-3359-4eac-9017-c659192f172f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.057348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 501/988] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:15.064122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:15.177572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930d38b-7fd1-4cf1-b16a-17743f18576d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:15.210751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:15.211360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:15.211654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:15.211929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:15.212710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] RBAC: Adding query filter params (role.id=5bd19a2337764593906955917f1ac831, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=f2ee302ccbdb4fa29fefc821567048e5) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:15.213042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:15.214461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12cc04-537c-484a-8bdc-e9b7023d0898 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:15.221015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 488/989] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:15 2022] GET /identity/v3/role_assignments?user.id=f2ee302ccbdb4fa29fefc821567048e5&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=5bd19a2337764593906955917f1ac831 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:16.580237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a860-b685-42f4-9808-ff71f7572419 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.580784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a860-b685-42f4-9808-ff71f7572419 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.580784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a860-b685-42f4-9808-ff71f7572419 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.582344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 502/990] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:16.587216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bbb6f-0884-46fd-ab96-88b7f9559419 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.587588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bbb6f-0884-46fd-ab96-88b7f9559419 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.587770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3bbb6f-0884-46fd-ab96-88b7f9559419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.606632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1c3bbb6f-0884-46fd-ab96-88b7f9559419 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:16.607824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1c3bbb6f-0884-46fd-ab96-88b7f9559419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:16.611665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.678297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 489/991] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:16.701727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a860-b685-42f4-9808-ff71f7572419 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.755100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:16.810050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a860-b685-42f4-9808-ff71f7572419 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.827818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f18dd-51d7-42a4-9e70-d87dbe3c7048 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.828126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f18dd-51d7-42a4-9e70-d87dbe3c7048 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.828384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f18dd-51d7-42a4-9e70-d87dbe3c7048 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.829934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 490/992] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:16.834793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.835021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.835303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.844515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:16.845182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:16.845467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:16.845741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] PATH_INFO: `/v3/services/compute` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:16.846366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:16.847729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:16.853172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:16.854359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:16.856753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:16.857711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:16.859039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:16.861124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 503/993] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:16.864768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.927179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 491/994] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:16.950015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:16.974362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea474007-5299-4564-bc6c-ddb683130bed None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:16.988073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.012317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.013270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.013554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.013836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.014546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.014847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.016222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.021262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 504/995] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:17.023141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.026657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.056028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b72eb-3177-4883-8c6d-c191a48618de None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.100892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.101505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.101829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.102061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.102689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.104049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:17.113889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:17.114898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:17.115726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 492/996] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Jul 27 19:56:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:17.123161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.142345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c1e07-e141-44aa-95b1-6e6b1ff70c5f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.178477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.179094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.179385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.179640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.180321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.180625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.182031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.187912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 505/997] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:17.193757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.235308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb48140-359c-4009-958a-d9a29f1af89d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.239305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.272843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.273493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.273777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.274058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.274734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.275039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.276419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0080ad-59d9-48c6-9ee4-1a15f04b4bca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.277666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.289024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 493/998] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:17.290321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e37fa-7f62-4088-bdbb-823ba184f55d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.290605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e37fa-7f62-4088-bdbb-823ba184f55d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.290882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e37fa-7f62-4088-bdbb-823ba184f55d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.292136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 494/999] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:17.292818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e37fa-7f62-4088-bdbb-823ba184f55d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.305052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1b76-231f-4e23-9a82-371984cf9bd1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.338322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.338946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.339230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.339505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.340258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] RBAC: Adding query filter params (interface=public, service_id=f9a18330ad654f8bb0c3905f0dad1dc6, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.340569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.341977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877db088-2b58-425d-8301-a0c2449673f5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.348679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 506/1000] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/endpoints?service_id=f9a18330ad654f8bb0c3905f0dad1dc6&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:17.349857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.350149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.350427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.369443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:17.370540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:17.393807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e37fa-7f62-4088-bdbb-823ba184f55d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.427679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.428294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.428577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.428852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.429555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.429861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.431222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.435155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 507/1001] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:17.441382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 495/1002] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:17.441531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.443190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.472100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.513743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.538652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b533b2f-f998-45c3-b7c5-555349c08cd8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.553270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe5fc-5767-4933-aa74-4a2324896fe2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.573993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.574611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.574894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.575171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.575821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.577321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.583467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 508/1003] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:17.584256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.586819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.588233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.588233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.588233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.588990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.590142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.608057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 496/1004] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:17.715601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.721145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22f99a-570e-4f8c-8929-67f99276e539 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.769459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.770108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.770390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.770659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] PATH_INFO: `/v3/services/image` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.771282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.772652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:17.782396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:17.783427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:17.784290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 509/1005] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:17.791577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:17.900298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cbaf3-4399-463a-ba60-333564901205 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:17.924857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:17.938221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:17.938846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:17.939129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:17.939402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:17.940078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] RBAC: Adding query filter params (name=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:17.940383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:17.941778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cd5d2-e813-4d79-9d6b-01e0ac7a466c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:17.947147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 497/1006] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:17.953220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:18.057993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed636c70-8741-495d-8862-c2af044cf477 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:18.106378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:18.107003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:18.107280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:18.107550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:18.108327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:18.109163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:18.110531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988f266-791c-45ba-8875-d92c89645dd1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:18.122552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 510/1007] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:19.606685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112deadf-6dea-467c-8baa-492e52cff324 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.607203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112deadf-6dea-467c-8baa-492e52cff324 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.607281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112deadf-6dea-467c-8baa-492e52cff324 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.609543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 498/1008] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:19.614681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.615084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.615267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.633268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:19.634447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:19.651644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:19.679287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac396-ad5f-4bf9-b490-4f926a6e0a39 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.679683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac396-ad5f-4bf9-b490-4f926a6e0a39 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.679845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bac396-ad5f-4bf9-b490-4f926a6e0a39 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.681360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 499/1009] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:19.686481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.686960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.687243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.705730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:19.706810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 511/1010] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:19.707025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:19.729141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:19.776838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 500/1011] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:19.800153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:19.839576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360cb66-4f15-466f-ad97-c652de3d2dfb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:19.876027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:19.876586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.876869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.877171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.877800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:19.879173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:19.888130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:19.889150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:19.890179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 512/1012] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:19.895451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:19.909600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2df0d1-b4b6-4507-9fd7-e00dc4b25f8c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:19.914903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:19.927103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:19.948154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:19.948872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:19.949253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:19.949656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:19.950750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:19.952218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:19.954002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 501/1013] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:19.959431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:20.008254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1bf5b-8ea3-482f-b30f-4684fd50c87e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:20.045352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:20.046657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.046933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.047201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.047870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:20.048169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:20.049580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:20.054666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 513/1014] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:20.060106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:20.066853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e10d71-2f2b-459f-a62c-d370a93725ba None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:20.105959 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:20.106576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.106854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.107127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.107825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:20.110254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85a7cc-a588-42df-8501-f195fd6d59bd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:20.138181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 502/1015] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:19 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:56:20.146748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05a8b1-ed62-455d-8877-24cbcf7e2fbd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.147046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05a8b1-ed62-455d-8877-24cbcf7e2fbd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.147327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05a8b1-ed62-455d-8877-24cbcf7e2fbd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.148886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 503/1016] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:20.154469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.154825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.155606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.174036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:20.175203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:20.176861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:20.199605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10741312-6858-40e5-9cc1-d4ff6751431e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:20.213284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:20.241322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:20.242271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.242563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.242844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.243582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:20.243859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:20.245613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:20.252838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 514/1017] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Jul 27 19:56:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:20.259288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:20.260557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 504/1018] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:20.285957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:20.376846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf5f9-5873-4b34-a444-80592e0e695a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:20.397905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c55760-08fe-4a3e-9f49-ff335d8b36e5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:20.410121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:20.410747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.411025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.411299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.412033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:20.413441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fe946-db14-40b9-bbd9-f5d8cd4937b7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:20.430280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:20.436599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 515/1019] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:20.450591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:20.451582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:20.452597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:20.453397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:20.454351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:20.456006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b92fc-21be-4fa4-bd08-4243f527dfb7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:20.474753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 505/1020] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:20 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:22.242474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4fab6-4012-42c1-9216-7ae24dab3f85 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.243006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4fab6-4012-42c1-9216-7ae24dab3f85 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.243006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4fab6-4012-42c1-9216-7ae24dab3f85 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.244438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 516/1021] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:22.250363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6020bee-5832-4b1b-a018-ad534ceea852 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.250771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6020bee-5832-4b1b-a018-ad534ceea852 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.250949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6020bee-5832-4b1b-a018-ad534ceea852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.269333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d6020bee-5832-4b1b-a018-ad534ceea852 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:22.270523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d6020bee-5832-4b1b-a018-ad534ceea852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:22.288274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:22.349734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 506/1022] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:22.374381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4fab6-4012-42c1-9216-7ae24dab3f85 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:22.479357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4fab6-4012-42c1-9216-7ae24dab3f85 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:22.505773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def63b8-7f90-475c-bd2e-7cddb47d81b3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.506106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def63b8-7f90-475c-bd2e-7cddb47d81b3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.506341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def63b8-7f90-475c-bd2e-7cddb47d81b3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.507827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 507/1023] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:22.512604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c5de-a212-4c44-b155-d252448de4e1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.512888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c5de-a212-4c44-b155-d252448de4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.513196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4c5de-a212-4c44-b155-d252448de4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.516524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:22.517393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.517821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.518258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.519552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:22.521176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:22.522792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 517/1024] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:22.529009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:22.533996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-88b4c5de-a212-4c44-b155-d252448de4e1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:22.535188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-88b4c5de-a212-4c44-b155-d252448de4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:22.596251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:22.606172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 508/1025] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:22.607179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211a540-7767-4eb3-ab36-d9445bcd2b6d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.607460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211a540-7767-4eb3-ab36-d9445bcd2b6d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.607732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211a540-7767-4eb3-ab36-d9445bcd2b6d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.609182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 509/1026] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:22.614824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.615118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.615561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.634477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:22.635673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:22.636838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89520-93d6-4b5a-b5e0-9619de756587 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:22.679111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:22.679722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.680012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.680282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.680958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:22.682412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:22.705723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 510/1027] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:22.706569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:22.716935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 518/1028] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:22 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 19:56:22.722838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:22.731383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:22.834735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7008-8cc2-439a-b686-2779cf0f748d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:22.854720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce56d23-ab68-4b36-918a-621d867f59b1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:22.878945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:22.879850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.880367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.880883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] PATH_INFO: `/v3/services/image` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.881554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:22.882959 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:22.886652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:22.894902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:22.895957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:22.896829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:22.897933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 511/1029] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:22 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:22.904739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:22.910148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:22.910934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:22.911312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:22.911593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:22.927697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:22.937169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:22.938278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:22.940298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 519/1030] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:22.947901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.002877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.041861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e20f0-4b45-4a6e-b144-c83e38452664 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.072653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f5ed9-d85c-4f6c-ae1f-caca4e26d921 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.095699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.096335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.096914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.097226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.097927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] RBAC: Adding query filter params (name=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.098235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.099626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.105084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 512/1031] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:23.112187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.116958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.117619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.117966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.118241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.119019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.119324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.120713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.133206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 520/1032] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:23.138633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.196206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.225524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ad8b-d751-40ec-b997-502383b13a1c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.260453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c68da-6b0b-4244-a508-28f339047d44 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.262679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.263339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.263609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.263851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.265233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] RBAC: Adding query filter params (type=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.265791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.267184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.269472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.277126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 513/1033] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:23.282653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.309072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.309728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.310080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.310367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.322299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.323772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:23.343135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:23.344110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:23.345171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:23.345171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:23.345171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:23.345171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:23.345171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 521/1034] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:23.351700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.418113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b095b8-3570-42cc-8087-648ebd9b3900 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.460812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.461474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.461755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.462028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.462782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] RBAC: Adding query filter params (interface=public, service_id=73a2ed94550142bda890a2ce3ad0e7b2, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.463089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.464513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.469889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.476871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 514/1035] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/endpoints?service_id=73a2ed94550142bda890a2ce3ad0e7b2&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:23.477105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69358c-15e9-43d4-b5cc-319272bab4d3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.478546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.515854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.516484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.516764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.517067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.517771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.518077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.519451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.537441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 522/1036] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:23.539364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.559482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.611081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b2a5c-a06a-41d4-b6c6-386de19b700d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.653133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.653767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.654052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.654329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.655706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.657241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.673112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 515/1037] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/services => generated 1957 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:56:23.679065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e003a0d-b1ef-4782-a635-129ee7578845 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.721981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.722656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.722939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.723215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.731568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.733194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:23.746283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:23.747436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:23.748402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:23.752784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 523/1038] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:23.753854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.865491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533344f-a689-4a3f-8073-e8550c9fa56d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:23.892397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:23.901137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:23.901846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:23.902197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:23.902542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:23.903410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:23.903790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:23.905868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:23.924767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 516/1039] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:23.930653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:23.951435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.039648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b4d06-a53f-4f82-a6a0-8e045b7fe42b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.078371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.078998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.079280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.079552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.151613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.153452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.173418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.259774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 524/1040] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:24.265423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.369312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254df33-2a32-4c26-bb54-3b334072fbc8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.404735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.405451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.405883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.406170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.416706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.418199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:24.434174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:24.435113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:24.441534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:24.441534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:24.441534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:24.441534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:24.441534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 517/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:24.441827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.552659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ae78c-f69b-42da-a1ba-af934109fd9d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.571244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.589024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.589645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.589926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.590201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.590889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:24.591193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.593424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.609401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 525/1042] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:24.610664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.651001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:24.713865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0a7bf-2210-46ef-855a-ffb8688d4fbc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.748891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.749577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.749860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.750139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.756866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.758341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:24.769944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:24.770871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:24.777952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 518/1043] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:24.778446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.886942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15baa9f-7c19-488a-83ba-63b9e8f74900 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:24.922648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:24.923266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.923547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.923821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.924574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:24.924883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:24.926359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e34a5e-8284-49b4-8b4c-0332ae100a47 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:24.944268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 526/1044] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:24.945518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:24.952920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e7772-4951-413d-9500-17bde63b9f94 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.953257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e7772-4951-413d-9500-17bde63b9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.953496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e7772-4951-413d-9500-17bde63b9f94 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.954826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 527/1045] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:24.959816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:24.960103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:24.960380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:24.979917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:24.981141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:25.016171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.051470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.056001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f13bb7-06ce-4020-90d8-b665d686bcea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.057921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 528/1046] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:25.082214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.104647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.105315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.105605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.105887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.106687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] RBAC: Adding query filter params (role.id=5bd19a2337764593906955917f1ac831, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:25.107077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:25.109018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:25.115182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 519/1047] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:24 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=5bd19a2337764593906955917f1ac831 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:25.203894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739dd6d-186b-43f3-b89f-7dd0d8eee283 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.242187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.243138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.243569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.243991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.256441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:25.262856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:25.263870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:25.265171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 529/1048] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:25.272269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.380188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556dd9-c4f9-4a35-b61c-0c1666daf4c2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.414838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.415458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.415741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.416016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.416727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:25.417060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:25.418439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:25.427413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 520/1049] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:25.432897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.463318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.536825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbd310-af7e-40bb-b716-45ba2a63b046 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.573518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.574162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.574444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.574721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] PATH_INFO: `/v3/users/d487f4afd5894547bcf9447cd8eb747b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.592004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] RBAC: Authorizing `identity:get_user(user_id=d487f4afd5894547bcf9447cd8eb747b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:25.594147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd46051-2d9f-4e9b-b253-ac4007b0e375 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:25.600555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 530/1050] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/users/d487f4afd5894547bcf9447cd8eb747b => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:25.602112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.628805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.691910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df19c-dfda-47ba-a155-701c95c8e1dc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.692507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df19c-dfda-47ba-a155-701c95c8e1dc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.692785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df19c-dfda-47ba-a155-701c95c8e1dc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.694248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 531/1051] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:25.698721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.699006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.699285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.708142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0fa35-0694-4fa7-bd62-eb55e8c369f6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.719017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:25.720132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:25.743665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.744290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.744573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.744851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.787710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 532/1052] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:25.810391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.817864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:25.819341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:25.829164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.848956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:25.924661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b66c4-a22f-45cd-8ab6-43e8b30459f4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:25.932280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 521/1053] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:25.937735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:25.960170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:25.960812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:25.961129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:25.961408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:25.962030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:25.963399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:25.972947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:25.973939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:25.974761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 533/1054] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:25.979810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.043161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99664c1f-002f-45bc-a7e0-38616b14eefe None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.063598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:26.076751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:26.086590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d86338-4167-4d09-bea8-ff8ea4d2d1ae None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.093183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.094040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.094316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.094965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.095789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.097243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.098901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 522/1055] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:26.104657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.123790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.124409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.124690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.124994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.125665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:26.125971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.127334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.132747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 534/1056] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:26.138010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.214158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc40d8-bcb5-4f90-a504-6cf69cdadbe5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.249717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75080973-05cb-48a6-8c15-2fa0e4d6f4a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.251435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.252057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.252340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.252611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.253438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=4e59dc666ab4440da0ba3840de14a93f, user.id=d487f4afd5894547bcf9447cd8eb747b) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:26.253749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.255176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.260763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 523/1057] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:26 2022] GET /identity/v3/role_assignments?user.id=d487f4afd5894547bcf9447cd8eb747b&scope.project.id=4e59dc666ab4440da0ba3840de14a93f&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:26.298320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.299131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.299487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.299832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.300626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] RBAC: Adding query filter params (type=image) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:26.301028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.302518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08532ae3-58d9-4110-9676-5e654047fb20 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.308535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 535/1058] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:26.315260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:26.375704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:26.424915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d8a28-4649-4962-9f10-2b8bd25ea843 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:26.461267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:26.461888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:26.462169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:26.462444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:26.463162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:26.464524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b462d8c-ce30-4ba3-85ab-acdb5b9b8e9c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:26.486132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 524/1059] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:56:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:27.115298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd2104-2b98-4919-9a87-3761ace8ee98 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.115824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd2104-2b98-4919-9a87-3761ace8ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.115824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd2104-2b98-4919-9a87-3761ace8ee98 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.117401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 536/1060] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:27.122307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.122598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.122874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.141457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:27.142641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:27.217326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 525/1061] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:27.239372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd2104-2b98-4919-9a87-3761ace8ee98 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.251610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.341870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd2104-2b98-4919-9a87-3761ace8ee98 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.377622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.378270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.378549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.378821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.389873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:27.394722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:27.395869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:27.397043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:27.400805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 537/1062] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:27.401834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.441059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.502324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf80c63-409d-421d-b1f7-66008c90fb97 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.536718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.537364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.537649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.537919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.538624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:27.538927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.540281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.553268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 526/1063] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:27.554443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.623795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.660730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd83a22-7b04-4f9f-bb9c-73c3234cde83 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.696265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.696905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.697226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.697502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.708700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.710178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:27.724251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:27.725423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:27.726340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:27.726340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:27.726340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:27.726340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:27.726340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 538/1064] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:27.732458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.831760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:27.835408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead13be9-68b1-4916-8535-be7d621dbde9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:27.871031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:27.871660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:27.871941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:27.872212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:27.872895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:27.873228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:27.874954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:27.891453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 527/1065] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:27.892706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:27.993399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0ef0e-dd57-4596-974f-65b7b7a6871d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.012654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.028354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.029027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.029312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.029589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.038200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.039643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:28.051213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:28.052178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:28.053202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 539/1066] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:28.059544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.169724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61343f4-40ef-4c9a-a2b5-3632f04731ce None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.205209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.205837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.206120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.206397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.207196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:28.207505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.208938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81c5f-7564-43a7-88c4-2793a8d90801 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.231146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 528/1067] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:28.232754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.292621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df722-963b-4353-9381-0cd3d9a6aa86 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.292912 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df722-963b-4353-9381-0cd3d9a6aa86 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.293221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2df722-963b-4353-9381-0cd3d9a6aa86 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.294692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 529/1068] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:28.300011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.300305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.300586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.319318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:28.320527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:28.333991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.340787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c14cce-0f37-4298-b2c8-c663f012dec1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.377515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.378164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.378447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.378721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.390447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.392072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.392668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 530/1069] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:28.407523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:28.408448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:28.409374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:28.409374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:28.409374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:28.409374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:28.414698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 540/1070] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:28.415144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.415575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.451422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.527979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28ec8f-b6f0-483b-b99d-a1e0a2c6e6af None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.528979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c54f0-44af-49ce-bdd5-98ca9d5ea306 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.561980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.562578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.562863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.563140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.563582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.563829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:28.564133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.564223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.564491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.564765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.565543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52831a10-3dac-40f2-ba2d-7d9bade31df2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.575458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:28.577532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 541/1071] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:28.578734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53ec1-8047-4cc8-b102-87bc12b79243 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.579020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53ec1-8047-4cc8-b102-87bc12b79243 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.579297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53ec1-8047-4cc8-b102-87bc12b79243 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.580693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 542/1072] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c1551d4e-6f24-4e60-a842-d3a9aeb71187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:28.580801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:28.587616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:28.588479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 531/1073] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:28.589214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db53ec1-8047-4cc8-b102-87bc12b79243 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.589287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.589287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.589287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.610968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:28.612189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:28.615176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.678995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.680736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 532/1074] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:28.681864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.689380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db53ec1-8047-4cc8-b102-87bc12b79243 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.727736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.728290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.728569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.728839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.735506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.736952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:28.748130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:28.749264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 543/1075] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:28.750171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.790415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c8ebb-11df-4c7d-8568-e94f5fb08c5c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.824631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.825281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.825559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.825830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.827262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:28.827680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.829307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.834698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.839365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 533/1076] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:28.840762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:28.854153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d077b-e2b4-492a-ab8a-37d1e266be07 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.887998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.888608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.888886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.889735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.913260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:28.947096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a0b63-e7e6-46a1-baba-c4c450e1447b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:28.958555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.960023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.981763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:28.982305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:28.982588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:28.982894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:28.983789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:28.984095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:28.985540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:28.996901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 534/1077] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:28.997559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.055088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.062503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 544/1078] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:29.063329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.098057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4444e-ebe7-4cfd-93cc-c30747a902d2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.119803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.132172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.132816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.133139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.133429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] PATH_INFO: `/v3/users/d487f4afd5894547bcf9447cd8eb747b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.134610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] RBAC: Authorizing `identity:get_user(user_id=d487f4afd5894547bcf9447cd8eb747b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.136039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.137682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 535/1079] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:28 2022] GET /identity/v3/users/d487f4afd5894547bcf9447cd8eb747b => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:29.138438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.172715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891e6e9-3e93-4899-b123-6bda6700a011 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.206635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.207297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.207578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.207851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/5bd19a2337764593906955917f1ac831` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.210077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=056e9b8fcfc84ee68a018f331e805582, role_id=5bd19a2337764593906955917f1ac831)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.211502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.223959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 545/1080] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:29 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/056e9b8fcfc84ee68a018f331e805582/roles/5bd19a2337764593906955917f1ac831 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:29.224747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.242353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3413d-847a-4c7f-9cbb-04b203f047fc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.276170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.289442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.290108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.290395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.290672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.303238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.304720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:29.321065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:29.322068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:29.322952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:29.322952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:29.322952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:29.322952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:29.322952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 536/1081] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:29.328535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.341566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f51c7a-b084-454a-af86-8e6156a91bad None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.358979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:29.390227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.390882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.391163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.391434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.392601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.394079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.395756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 546/1082] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:29 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:29.402367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:29.443797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634445d5-9e2d-49d4-9c89-ee6984bff251 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.478063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.478684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.479012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.479237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.479925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:29.480236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.481631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d69ca-c638-45bb-9c49-e2252047c59c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.494534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 537/1083] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:29.519565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e05c18-db33-408a-826e-ac9530db50fe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:29.559085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:29.559784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:29.560066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:29.560346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f/users/d487f4afd5894547bcf9447cd8eb747b/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:29.563030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] RBAC: Authorizing `identity:create_grant(project_id=4e59dc666ab4440da0ba3840de14a93f, user_id=d487f4afd5894547bcf9447cd8eb747b, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:29.564505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f883fa1-3350-4530-92e5-3d3989fa033d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:29.579824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 547/1084] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:29 2022] PUT /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f/users/d487f4afd5894547bcf9447cd8eb747b/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:56:31.260584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b898bf-a6e1-4135-b57d-c82a94a95f4a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.261137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b898bf-a6e1-4135-b57d-c82a94a95f4a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.261223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b898bf-a6e1-4135-b57d-c82a94a95f4a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.262698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 538/1085] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:31.267763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.268145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.268453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.286474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:31.287670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:31.322539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.356473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 548/1086] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:31.380245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b898bf-a6e1-4135-b57d-c82a94a95f4a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.423293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.482763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b898bf-a6e1-4135-b57d-c82a94a95f4a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.516190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:31.516827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.517145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.517423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.527269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6547af4a-9bee-44fe-ba53-a142126c8101 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:31.532989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:31.533980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:31.534842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 539/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:31.540128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.593772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e69f60-8fe2-43c5-bf8f-7b435dc8d7f8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.594062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e69f60-8fe2-43c5-bf8f-7b435dc8d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.594346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e69f60-8fe2-43c5-bf8f-7b435dc8d7f8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.595696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 540/1088] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 19:56:31.600498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.600784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.601097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.619802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:31.620923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:31.641609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94809aac-3106-4d00-87c7-6ab7389cdc2c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.674583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:31.675191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.675472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.675744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.676474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:31.676750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:31.678146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:31.686366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 549/1089] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:31.688249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 541/1090] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:56:31.692085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.711629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.723590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.741414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.792167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae74f2f-ed89-4bdf-8571-efb86e88d7ea None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.810459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d54ee-5ca9-4231-b4a0-8ed3af2222f0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.827591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:31.828221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.828497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.828769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.838922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:31.840355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:31.847308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:31.847944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.848229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.848503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:31.854987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:31.855857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:31.856997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:31.856997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:31.856997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:31.856997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:31.856997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 550/1091] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:31.859647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:31.861796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:31.864555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:31.865567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:31.866607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 542/1092] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:56:31.871579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:31.946169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.962470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7076c5c-e305-4367-b13c-45806283186d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.963156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:31.973767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9e335-bdbc-4282-97a9-7b832f0cfdfc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:31.996213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:31.996832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:31.997142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:31.997417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:31.998103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:31.998438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:31.999814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.007008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.007620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.007896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.008167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.008869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:32.009211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.010555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.011247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 551/1093] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:32.017504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.019918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 543/1094] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:32.025442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.117114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8a404-617f-4e3f-9a8a-0e3f5afaeac0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.134738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8b39-24d9-43a9-9639-8193e87d9070 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.151670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.152312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.152589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.152861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.160257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.161758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.168315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.168951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.169255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.169527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] PATH_INFO: `/v3/users/d487f4afd5894547bcf9447cd8eb747b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.170772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] RBAC: Authorizing `identity:get_user(user_id=d487f4afd5894547bcf9447cd8eb747b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.172178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:56:32.173279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:32.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 19:56:32.175207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:32.175746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 544/1095] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/users/d487f4afd5894547bcf9447cd8eb747b => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:32.175826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 552/1096] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:32.179561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.179895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.183379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.202891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.278294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9103-6514-463f-9a09-f68c390bc152 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.278846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b8b8-539e-4c99-92d4-2a9fad8eec1c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.312339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.312856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.313009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.313282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.313498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.313597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.313781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.314056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.314323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:32.314620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.316048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.332799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 553/1097] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:32.338549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.380790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.382278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.399351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.402285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.438959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e569b-6295-4e4c-8f35-a653a243f296 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.472775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.473416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.473704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.473983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.482464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 545/1098] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:32.488346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.539753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.541244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.584854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e396f-0bcb-4ac8-92ee-a12de922d4b3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.598774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.604767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.619871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.620513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.620792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.621103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.622257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.624285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.625990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 546/1099] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:56:32.631985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.644325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 554/1100] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 19:56:32.645338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.734278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64279f-e47d-4c94-adc5-9f6a95af4093 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.743267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00263cad-da27-4eb6-b8b4-7fbcf3b1a89a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.769515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.770131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.770411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.770685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.771474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=4e59dc666ab4440da0ba3840de14a93f, user.id=d487f4afd5894547bcf9447cd8eb747b) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:32.771785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.773230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.777548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.778185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.778469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.778747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.779007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 547/1101] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/role_assignments?user.id=d487f4afd5894547bcf9447cd8eb747b&scope.project.id=4e59dc666ab4440da0ba3840de14a93f&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:32.789758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.791247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:56:32.808814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:32.809994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:32.811612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:56:32.811612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:56:32.811612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:56:32.811612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:32.813306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 555/1102] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:32.816775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.872863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:32.922909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfb1ae-cc87-4ea0-8257-5d856253fe2e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:32.961089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:32.961726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:32.962005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:32.962283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:32.962975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:32.963280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:32.964647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:32.977064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 548/1103] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:32.982796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:32.999427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:33.085538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e218072-ee44-4f1f-b46d-ef402c5cf8cf None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:33.119097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:33.119737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:33.120012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:33.120282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:33.126741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:33.128165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:56:33.139248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:33.140291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:33.141239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 556/1104] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:33.146177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:33.247191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f1176-6455-4de0-a9f3-a12e3bad4462 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:33.260315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:33.282785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:33.283392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:33.283671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:33.283945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:33.284696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:33.285041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:33.286471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21408-5ec5-4c9f-86ac-c0bc154f8107 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:33.297866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 549/1105] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:33.303890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:33.377030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:33.402075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9925-02fd-4ca6-9aaf-83f42307c30d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:33.435604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:33.436895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:33.437205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:33.437483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:33.438267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] RBAC: Adding query filter params (role.id=5bd19a2337764593906955917f1ac831, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=056e9b8fcfc84ee68a018f331e805582) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:33.438579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:33.440005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa187373-4bc8-4d53-81bc-2c82dc54dc59 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:33.456998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 557/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:33 2022] GET /identity/v3/role_assignments?user.id=056e9b8fcfc84ee68a018f331e805582&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=5bd19a2337764593906955917f1ac831 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:34.822274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0bd2-e88d-4226-9a6c-f08c689b3a34 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.822799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0bd2-e88d-4226-9a6c-f08c689b3a34 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.822916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed0bd2-e88d-4226-9a6c-f08c689b3a34 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.824173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 550/1107] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:34.830221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:34.830593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:34.830779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:34.849023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:34.850224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:34.914978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 558/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:34.939271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed0bd2-e88d-4226-9a6c-f08c689b3a34 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.039076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed0bd2-e88d-4226-9a6c-f08c689b3a34 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.071730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.072427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.072648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.072924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.074535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.075525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00a2fa-3111-4658-9546-d7432b896366 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.077755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 551/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:35.083532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:35.147355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:35.179586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5062a-73ea-4d09-9187-262537d21539 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:35.213143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:35.213780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:35.214062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:35.214335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:35.215020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:35.216453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256721a-2fa0-4c86-9eb6-56b82ee2ff62 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:35.236933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 559/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:35 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:37.262149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84de2c-c1c4-4b4e-83d8-1f5260523f06 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.262700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84de2c-c1c4-4b4e-83d8-1f5260523f06 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.262700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84de2c-c1c4-4b4e-83d8-1f5260523f06 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.264579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 552/1111] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:37.269683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.270064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.270248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.288617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:37.289908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:37.353973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 560/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:37.376774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84de2c-c1c4-4b4e-83d8-1f5260523f06 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:37.420886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:37.478054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84de2c-c1c4-4b4e-83d8-1f5260523f06 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:37.512652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:37.513320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.513590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.514485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] PATH_INFO: `/v3/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.515969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] RBAC: Authorizing `identity:get_role(role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:37.517353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:37.518669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 553/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:37.524723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:37.585064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:37.630088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267193de-d3ba-471e-8307-aeff2353a8e0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:37.662261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:37.662886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.663160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.663430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] PATH_INFO: `/v3/users/e9582639f2b74fa4b1b85d960306e939` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.676493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] RBAC: Authorizing `identity:get_user(user_id=e9582639f2b74fa4b1b85d960306e939)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:37.677956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:37.679563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 561/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/users/e9582639f2b74fa4b1b85d960306e939 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:37.685412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:37.782727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7d899-4756-472f-a094-3a7c2e0b1a32 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:37.791048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:37.816849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:37.817735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:37.818012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:37.818285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:37.884593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:37.886081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:37.979074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 554/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:37.985511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.081957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3903a5-f0dd-4a36-be54-b9eaed714846 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.094899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:38.115395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.116033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.116311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.116603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.117750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.119179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978226d-4e9c-4423-8127-038ed19051a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.120836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 562/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:37 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:38.127416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:38.137240 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:38.223682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefc953-b71b-461c-8308-717a023db35d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:38.256044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:38.256642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:38.256917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:38.257216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:38.257994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=4e59dc666ab4440da0ba3840de14a93f, user.id=e9582639f2b74fa4b1b85d960306e939) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:38.258303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:38.259710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddaf33-201e-48b7-b986-90cc32a17ab3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:38.265528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 555/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:38 2022] GET /identity/v3/role_assignments?user.id=e9582639f2b74fa4b1b85d960306e939&scope.project.id=4e59dc666ab4440da0ba3840de14a93f&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:40.267031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53795618-b893-434d-b4ec-9bfc5c26cb11 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.267576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53795618-b893-434d-b4ec-9bfc5c26cb11 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.267576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53795618-b893-434d-b4ec-9bfc5c26cb11 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.268937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 563/1118] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:40.274191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.274586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.274752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.292892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:40.294111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:40.359348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 556/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:40.381876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53795618-b893-434d-b4ec-9bfc5c26cb11 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.480712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53795618-b893-434d-b4ec-9bfc5c26cb11 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:40.514582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:40.515212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.515497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.515776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] PATH_INFO: `/v3/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.517250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] RBAC: Authorizing `identity:get_role(role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:40.518623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:40.519953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 564/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:40 2022] GET /identity/v3/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:40.525763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.527931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:40.621696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f44196-fa3c-4a1e-b0be-c2f99afdb63d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:40.653561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:40.654184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.655085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.655363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] PATH_INFO: `/v3/users/e9582639f2b74fa4b1b85d960306e939` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.656513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] RBAC: Authorizing `identity:get_user(user_id=e9582639f2b74fa4b1b85d960306e939)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:40.657964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:40.659530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 557/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:40 2022] GET /identity/v3/users/e9582639f2b74fa4b1b85d960306e939 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:40.665412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.713621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:40.761585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d3cc-3a17-4598-898f-3df6d213bc23 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:40.794131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:40.794754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.795327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.795451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.796579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:40.798024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e609c0-fa10-49b5-b6b0-53048cf611e2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:40.799659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 565/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:40 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:40.805596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:40.872601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:40.903234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e294484-459d-4782-9d74-4941a8e76595 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:40.936280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:40.937002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:40.937291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:40.937568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f/users/e9582639f2b74fa4b1b85d960306e939/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:40.939734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] RBAC: Authorizing `identity:create_grant(project_id=4e59dc666ab4440da0ba3840de14a93f, user_id=e9582639f2b74fa4b1b85d960306e939, role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:40.941181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a44be-31c2-49a1-a559-c30a9db49c3e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:40.954293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 558/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:40 2022] PUT /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f/users/e9582639f2b74fa4b1b85d960306e939/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:56:42.949618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d383-bf70-4dbc-b67b-1b129ebed61c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:42.950173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d383-bf70-4dbc-b67b-1b129ebed61c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:42.950173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d383-bf70-4dbc-b67b-1b129ebed61c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:42.952322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 566/1124] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:42.957734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:42.958237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:42.958338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:42.976594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:42.977813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:43.043652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 559/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:43.067771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6d383-bf70-4dbc-b67b-1b129ebed61c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.166736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6d383-bf70-4dbc-b67b-1b129ebed61c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:43.183146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.201509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:43.202067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.202344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.202616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] PATH_INFO: `/v3/roles/596b1bffb8f24794aa2c5d9d294a6917` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.204024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] RBAC: Authorizing `identity:get_role(role_id=596b1bffb8f24794aa2c5d9d294a6917)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:43.205420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:43.206758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 567/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/roles/596b1bffb8f24794aa2c5d9d294a6917 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:43.212653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.269205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.309416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff2a50-3cd7-4912-bbd5-c09b28e19852 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:43.342691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:43.343311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.343593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.343868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] PATH_INFO: `/v3/users/e9582639f2b74fa4b1b85d960306e939` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.345039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] RBAC: Authorizing `identity:get_user(user_id=e9582639f2b74fa4b1b85d960306e939)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:43.346463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:43.348032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 560/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/users/e9582639f2b74fa4b1b85d960306e939 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:43.353758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.448889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f51db-0ba8-42f7-8255-e1a7c063b048 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:43.481807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:43.482997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.483445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.483719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.532164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.548190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:43.549662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:43.643471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 568/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:43.649559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.747566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4830d4-6c9e-4a6d-8f15-daeb8e23f468 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:43.773098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.782346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:43.782987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.783273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.783553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] PATH_INFO: `/v3/projects/4e59dc666ab4440da0ba3840de14a93f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.784703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] RBAC: Authorizing `identity:get_project(project_id=4e59dc666ab4440da0ba3840de14a93f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:43.786177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85897620-64d1-442f-866f-143aa4a3f624 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:43.787843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 561/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/projects/4e59dc666ab4440da0ba3840de14a93f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:43.794099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:43.873979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:43.889763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23ba9-5954-49d3-a85d-6d5b7d7bf12e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:43.921986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:43.922597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:43.922876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:43.923146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:43.923917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] RBAC: Adding query filter params (role.id=596b1bffb8f24794aa2c5d9d294a6917, scope.project.id=4e59dc666ab4440da0ba3840de14a93f, user.id=e9582639f2b74fa4b1b85d960306e939) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:43.924220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:43.925663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb66e4-d879-44a0-962b-dd39bc54b749 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:43.931398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 569/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:43 2022] GET /identity/v3/role_assignments?user.id=e9582639f2b74fa4b1b85d960306e939&scope.project.id=4e59dc666ab4440da0ba3840de14a93f&role.id=596b1bffb8f24794aa2c5d9d294a6917 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:45.972420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b50a-8a88-48c5-8aa4-f2aecdf52457 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:45.973044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b50a-8a88-48c5-8aa4-f2aecdf52457 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:45.973044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80b50a-8a88-48c5-8aa4-f2aecdf52457 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:45.974559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 562/1131] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:45.979708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:45.980156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:45.980259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.000148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:46.001370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:46.066458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 570/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:46.089687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b50a-8a88-48c5-8aa4-f2aecdf52457 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:46.188292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80b50a-8a88-48c5-8aa4-f2aecdf52457 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:46.220657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:46.221314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:46.221596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:46.221878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.223068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:46.224447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0525e-67cc-468c-9f7b-a9c607a007fa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:46.226041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 563/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:46.231815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:46.307836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:46.330125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d5b3d-e200-4046-a7a5-5eb4fa2d5aa7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:46.363083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:46.363688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:46.363964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:46.364235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:46.364986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:46.366405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a262f-3c12-40ee-9d51-06e4ebd72db1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:46.391316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 571/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:56:46 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:48.419514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482de10-4a9f-40d7-ab45-a7c761a169e2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.420039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482de10-4a9f-40d7-ab45-a7c761a169e2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.420039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482de10-4a9f-40d7-ab45-a7c761a169e2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.421450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 564/1135] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:48.426750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.427164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.427297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.445285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:48.446459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:48.509864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 572/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:48.534053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482de10-4a9f-40d7-ab45-a7c761a169e2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:48.596111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:48.641751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482de10-4a9f-40d7-ab45-a7c761a169e2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:48.674652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:48.675217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.675491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.675761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.676886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:48.678351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a99e79-7b9f-45d2-b57a-b17cc31fb7d2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:48.680075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 565/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:56:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:48.686306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:48.743175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:48.783500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a92244-93f4-4dd5-bf56-9ee9ad291414 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:48.817765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:48.818370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:48.818648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:48.818918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:48.819596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:48.821044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824d81e-9f4a-488b-adf5-efcbd427f408 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:48.841078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 573/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:56:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:56:50.864425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac289-2b14-4dce-a2b4-20b1aab52d61 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:50.864954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac289-2b14-4dce-a2b4-20b1aab52d61 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:50.865066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac289-2b14-4dce-a2b4-20b1aab52d61 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:50.866354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 566/1139] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:50.871406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:50.871814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:50.871969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:50.890879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:50.892057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:50.956193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 574/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:50.978625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ac289-2b14-4dce-a2b4-20b1aab52d61 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.078744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ac289-2b14-4dce-a2b4-20b1aab52d61 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.108318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.113518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.114155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.114489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.114718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.125147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:51.130057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:51.131200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:51.132298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 567/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:51.137317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.182114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.235439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4692c69-b430-4cb1-a8e7-ffca7c40293f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.267772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.268373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.268654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.268929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.269668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:51.269973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.271338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.279123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 575/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:51.284851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.381691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398374f-8b76-470a-8528-fc9ed10cf293 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.412739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.413383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.413660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.413930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] PATH_INFO: `/v3/users/834a5cb2deef4040b91d064b882377d3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.426827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] RBAC: Authorizing `identity:get_user(user_id=834a5cb2deef4040b91d064b882377d3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.428246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.429845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 568/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/users/834a5cb2deef4040b91d064b882377d3 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:51.436220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.532369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b13c0f-012a-4ce0-ba24-d9824bc12d8b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.540757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.565312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.565918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.566201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.566482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.631405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.632878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.732008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 576/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:51.739089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.787914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.837668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e620b21-a565-4d35-a324-c5ec1142dded None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:51.870038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:51.870674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:51.870956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:51.871232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] PATH_INFO: `/v3/projects/967f002090e94cf6969c7671e5b003a3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:51.872373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] RBAC: Authorizing `identity:get_project(project_id=967f002090e94cf6969c7671e5b003a3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:51.874438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81e444-edda-4f1c-84a9-47b5dfbbcfd1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:51.876083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 569/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/projects/967f002090e94cf6969c7671e5b003a3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:51.882173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:51.892481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:51.977144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a7c9b-51e5-4286-999b-ab812bb8d91e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:52.011185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:52.011781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:52.012058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:52.012330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:52.013129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=967f002090e94cf6969c7671e5b003a3, user.id=834a5cb2deef4040b91d064b882377d3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:52.013476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:52.014892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ba51b-d50a-425b-9dd4-27e4ff26d6a1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:52.020051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 577/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:51 2022] GET /identity/v3/role_assignments?user.id=834a5cb2deef4040b91d064b882377d3&scope.project.id=967f002090e94cf6969c7671e5b003a3&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:53.981962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66dc6b-5ad0-41e4-baae-0b1bfc4a5877 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:53.982511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66dc6b-5ad0-41e4-baae-0b1bfc4a5877 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:53.982605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66dc6b-5ad0-41e4-baae-0b1bfc4a5877 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:53.984450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 570/1147] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:53.989865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:53.990303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:53.990407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.007920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:54.009120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:54.073018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 578/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:54.096469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66dc6b-5ad0-41e4-baae-0b1bfc4a5877 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.195477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66dc6b-5ad0-41e4-baae-0b1bfc4a5877 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.212566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.229479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.230107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.230390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.230668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.240688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:54.245522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:54.246579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:56:54.247755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 571/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:54.252633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.254855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.349382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61eeaa9-f38f-422b-9a23-15c155f0d289 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.381287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.381893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.382175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.382453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.383162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:54.383469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.384936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.393393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 579/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.399164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.496391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128703-8b7c-4631-8c1c-db1105972247 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.530049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.530668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.530948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.531224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] PATH_INFO: `/v3/users/834a5cb2deef4040b91d064b882377d3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.532372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] RBAC: Authorizing `identity:get_user(user_id=834a5cb2deef4040b91d064b882377d3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.533831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.535400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 572/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/users/834a5cb2deef4040b91d064b882377d3 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.540949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.601593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.638086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932a80d-78e8-49e2-8997-3ce7521e9b28 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.670095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.670783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.671062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.671339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] PATH_INFO: `/v3/projects/967f002090e94cf6969c7671e5b003a3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.672468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] RBAC: Authorizing `identity:get_project(project_id=967f002090e94cf6969c7671e5b003a3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.673906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e6778-6ef8-4232-aab7-aa0408116815 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.675539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 580/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:54 2022] GET /identity/v3/projects/967f002090e94cf6969c7671e5b003a3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:54.681313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:54.726663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:54.777883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8867f-ad53-44ca-9f49-c50c7472c8b6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:54.810922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:54.811572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:54.811854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:54.812131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] PATH_INFO: `/v3/projects/967f002090e94cf6969c7671e5b003a3/users/834a5cb2deef4040b91d064b882377d3/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:54.814062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] RBAC: Authorizing `identity:create_grant(project_id=967f002090e94cf6969c7671e5b003a3, user_id=834a5cb2deef4040b91d064b882377d3, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:54.815493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c361af-585d-4ad3-9d50-8c45f24a8176 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:54.828701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 573/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:56:54 2022] PUT /identity/v3/projects/967f002090e94cf6969c7671e5b003a3/users/834a5cb2deef4040b91d064b882377d3/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 19:56:56.817864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6a244-89af-4f92-8f14-67f409fc036a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:56.818417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6a244-89af-4f92-8f14-67f409fc036a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:56.818417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6a244-89af-4f92-8f14-67f409fc036a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:56.819776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 581/1154] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:56.824982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:56.825417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:56.825578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:56.844159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:56:56.845371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:56:56.910022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 574/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:56:56.933153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6a244-89af-4f92-8f14-67f409fc036a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.033181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6a244-89af-4f92-8f14-67f409fc036a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.061383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.066417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.067047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.067582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.067857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.079223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:56:57.084160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:56:57.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:56:57.086080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 582/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:56:57.091389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.124403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.187103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0a0cc-d9b8-47d7-a2ff-5549e3d257ee None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.219416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.220026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.220301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.220569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.221492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:57.221794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.223148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.230758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 575/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.237143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.334913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1b7da-f8f8-4d65-8e36-7f075c385b59 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.367065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.367686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.367971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.368249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] PATH_INFO: `/v3/users/834a5cb2deef4040b91d064b882377d3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.369430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] RBAC: Authorizing `identity:get_user(user_id=834a5cb2deef4040b91d064b882377d3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.370867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.372497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 583/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/users/834a5cb2deef4040b91d064b882377d3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.378693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.473610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.477548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef468-6455-4810-b980-17d7d65715ec None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.509899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.510504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.510788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.511066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.577209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.578646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.668842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.677026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 576/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:56:57.679325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.739339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:56:57.781264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb1327-2212-4ad0-b932-53221024f548 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.815796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.816844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.817557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.817843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] PATH_INFO: `/v3/projects/967f002090e94cf6969c7671e5b003a3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.819014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] RBAC: Authorizing `identity:get_project(project_id=967f002090e94cf6969c7671e5b003a3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.820452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac4a23-68fb-4ef9-af2e-05b6dea1f154 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.822163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 584/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/projects/967f002090e94cf6969c7671e5b003a3 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:57.829333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:56:57.923558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6aa51-0853-470e-a977-0037342b48f7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:56:57.956757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:56:57.957388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:57.957666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:57.957936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:57.958720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=967f002090e94cf6969c7671e5b003a3, user.id=834a5cb2deef4040b91d064b882377d3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:56:57.959029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:56:57.960438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0348bd-086c-4ed7-83d2-e26ac20c8722 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:56:57.966118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 577/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:56:57 2022] GET /identity/v3/role_assignments?user.id=834a5cb2deef4040b91d064b882377d3&scope.project.id=967f002090e94cf6969c7671e5b003a3&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:56:59.992621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560f96-e7b7-4da5-bce1-45c42d65d3d2 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:56:59.993209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560f96-e7b7-4da5-bce1-45c42d65d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:56:59.993209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560f96-e7b7-4da5-bce1-45c42d65d3d2 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:56:59.994574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 585/1162] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:56:59.999680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.000095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.000238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.018010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:00.019198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:00.075658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.086255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 578/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:57:00.109357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560f96-e7b7-4da5-bce1-45c42d65d3d2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.210136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560f96-e7b7-4da5-bce1-45c42d65d3d2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.235282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.243494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.244044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.244323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.244594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.308220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.309689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.402730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 586/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:57:00.408835 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.505494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5564c81-990d-4c03-bd8c-670ecf7317fe None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.537582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.538209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.538492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.538768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.545991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.547437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bc6c9ed0-74ea-46b7-b24d-48fd795a031c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:57:00.558639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:57:00.559926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:57:00.561120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 579/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:57:00.565914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:00.593020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:00.662371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5b939-a277-4b4a-94a6-b8f5bca276fd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:00.699242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:00.699860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:00.700140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:00.700413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:00.701196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:00.701501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:00.702864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc56ac-1780-41ab-83b6-514e83176243 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:00.709781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 587/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Jul 27 19:57:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:57:02.702996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829777c-d247-407b-88e7-dbc707845966 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:02.703526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829777c-d247-407b-88e7-dbc707845966 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:02.703526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829777c-d247-407b-88e7-dbc707845966 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:02.704895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 580/1167] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:02.710115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbbab4-bce7-447f-b7a5-24c0b22f989b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:02.710513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbbab4-bce7-447f-b7a5-24c0b22f989b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:02.710673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbbab4-bce7-447f-b7a5-24c0b22f989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:02.728783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-22fbbab4-bce7-447f-b7a5-24c0b22f989b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:02.729983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-22fbbab4-bce7-447f-b7a5-24c0b22f989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:02.792412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 588/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:57:02.815310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829777c-d247-407b-88e7-dbc707845966 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:02.868133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:02.915367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829777c-d247-407b-88e7-dbc707845966 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:02.948005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:02.948608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:02.948890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:02.949200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:02.949900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:02.952369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c09c6-5dc5-4e35-be7f-2e92ae7260f8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:02.967600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 581/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Wed Jul 27 19:57:02 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:57:04.997893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943936-eb63-421f-a28c-fbd8b360ca38 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:04.998450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943936-eb63-421f-a28c-fbd8b360ca38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:04.998450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5943936-eb63-421f-a28c-fbd8b360ca38 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:04.999811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 589/1170] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:05.005036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:05.005429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:05.005581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:05.023758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:05.024982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:05.089738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 582/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 19:57:05.113285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5943936-eb63-421f-a28c-fbd8b360ca38 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:05.137331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:05.213900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5943936-eb63-421f-a28c-fbd8b360ca38 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:05.247679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:05.248393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:05.248629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:05.248904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] PATH_INFO: `/v3/domains/4a31ee7c70ab45cd9e341e4dc8202a4c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:05.259335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a31ee7c70ab45cd9e341e4dc8202a4c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:05.260781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f4293-fbc0-424a-91da-cdd1a0d32990 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:05.262426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 590/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Jul 27 19:57:05 2022] GET /identity/v3/domains/4a31ee7c70ab45cd9e341e4dc8202a4c => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:05.268797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:05.310347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:05.365127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc015f-bcb7-4634-b46d-324aca801ac1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:05.402398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:05.403008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:05.403289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:05.404180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:05.405073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:05.406488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f0708-4ef6-4fb5-80ec-857ff2a3aba2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:05.432323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 583/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:57:05 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:57:07.455549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e8a46-0a39-4ced-abee-966b04ca6dc1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:07.456068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e8a46-0a39-4ced-abee-966b04ca6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:07.456185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e8a46-0a39-4ced-abee-966b04ca6dc1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:07.457480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 591/1174] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:07.462725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:07.463141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:07.463287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:07.482265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:07.483454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:07.548026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 584/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:57:07.573126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e8a46-0a39-4ced-abee-966b04ca6dc1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:07.652651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:07.672319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e8a46-0a39-4ced-abee-966b04ca6dc1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:07.704842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:07.705499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:07.705827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:07.706055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] PATH_INFO: `/v3/domains/4a31ee7c70ab45cd9e341e4dc8202a4c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:07.707188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a31ee7c70ab45cd9e341e4dc8202a4c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:07.708615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d4d796-c4c0-4bc4-aedf-005d7aff149d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:07.710237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 592/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Jul 27 19:57:07 2022] GET /identity/v3/domains/4a31ee7c70ab45cd9e341e4dc8202a4c => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:07.716721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:07.743065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:07.816792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2558b7-7e98-4cc7-a817-85c55a8f880b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:07.849450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:07.850062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:07.850344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:07.850619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:07.851302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:07.852758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e68129-f765-4f9d-a3af-0b028a53ff9a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:07.874665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 585/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:57:07 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:57:09.883981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9c80-eea8-457e-941a-ebd354e9c75c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:09.884581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9c80-eea8-457e-941a-ebd354e9c75c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:09.884581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e9c80-eea8-457e-941a-ebd354e9c75c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:09.885934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 593/1178] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:09.891167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:09.891591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:09.891697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:09.911579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:09.912763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:09.978088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 586/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:57:10.001093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e9c80-eea8-457e-941a-ebd354e9c75c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:10.100487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e9c80-eea8-457e-941a-ebd354e9c75c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:10.133585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:10.134264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:10.134493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:10.134762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:10.145453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:57:10.150252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:57:10.156939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:57:10.158135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 594/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:57:10.159049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:10.169751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.254618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616753e-9da2-4b43-895a-b164d11af10c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:10.286092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:10.286691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:10.286968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:10.287239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:10.287937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:10.288236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:10.289624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:10.297827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 587/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:10.303492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:10.316361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.399863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5616216-6c58-4096-975c-ddf31c50e18b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:10.432827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:10.433487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:10.433771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:10.434048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] PATH_INFO: `/v3/users/aafe5ca0821e4120a826f2243794d54b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:10.447846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=aafe5ca0821e4120a826f2243794d54b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:10.449315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:10.450949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 595/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/users/aafe5ca0821e4120a826f2243794d54b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:10.456609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:10.525159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.555094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954477-1ae2-4a48-bca3-fe45c2b5dd7f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:10.587689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:10.588294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:10.588579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:10.588858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:10.654520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:10.655892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:10.733814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.764416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 588/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:57:10.770306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:10.835056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:10.866773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6709e-125b-438b-9f04-7fef84d3d7b7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:10.898905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:10.899526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:10.899801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:10.900071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] PATH_INFO: `/v3/projects/69ea706106c347c59e61a322c7183a7f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:10.901218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=69ea706106c347c59e61a322c7183a7f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:10.902671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f050-a8b3-4412-a2fb-890fa8da3ac1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:10.905013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 596/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/projects/69ea706106c347c59e61a322c7183a7f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:10.911083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:11.006354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4555c-9476-439c-98af-4a49aef4fe94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:11.040272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:11.040873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:11.041307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:11.041587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:11.042374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=69ea706106c347c59e61a322c7183a7f, user.id=aafe5ca0821e4120a826f2243794d54b) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:11.042685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:11.044170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c84ea4-43f8-43ec-80d5-fbefbcb843df None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:11.049535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 589/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:57:10 2022] GET /identity/v3/role_assignments?user.id=aafe5ca0821e4120a826f2243794d54b&scope.project.id=69ea706106c347c59e61a322c7183a7f&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:13.016815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce851-969a-48ce-851c-b69f83f418fb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.017403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce851-969a-48ce-851c-b69f83f418fb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.017403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce851-969a-48ce-851c-b69f83f418fb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.018789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 597/1186] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:13.023941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.024357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.024490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.042720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:13.044017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:13.071828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:13.111809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 590/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:57:13.133812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce851-969a-48ce-851c-b69f83f418fb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:13.232715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce851-969a-48ce-851c-b69f83f418fb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:13.266469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:13.267143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.267372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.267649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.270248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:13.278099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:57:13.283668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:57:13.284564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:57:13.285470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 598/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:57:13.290773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:13.387390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e81c52-ac32-494e-bc36-0b5f389077bd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:13.420096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:13.420702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.421001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.421280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.421984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:13.422286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:13.423644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:13.431352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 591/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:13.437832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:13.534864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfdb67-e151-485e-9ddb-d41eea6f0f18 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:13.566689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:13.567570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.567845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.568115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] PATH_INFO: `/v3/users/aafe5ca0821e4120a826f2243794d54b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.569289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] RBAC: Authorizing `identity:get_user(user_id=aafe5ca0821e4120a826f2243794d54b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:13.570727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:13.572310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 599/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:57:13 2022] GET /identity/v3/users/aafe5ca0821e4120a826f2243794d54b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:13.578168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:13.606043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:13.675497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589489e-dfed-4b9d-9e73-af70a1d415f6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:13.708493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:13.709161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.709441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.709715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] PATH_INFO: `/v3/projects/69ea706106c347c59e61a322c7183a7f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.710850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] RBAC: Authorizing `identity:get_project(project_id=69ea706106c347c59e61a322c7183a7f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:13.712269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0513127-8287-4f02-badf-3982df891f4d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:13.713941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 592/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:57:13 2022] GET /identity/v3/projects/69ea706106c347c59e61a322c7183a7f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:13.720908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:13.781024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:13.818621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285f99f-edb1-4f6e-b2d7-662f8958bff3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:13.851152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:13.851804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:13.852086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:13.852364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] PATH_INFO: `/v3/projects/69ea706106c347c59e61a322c7183a7f/users/aafe5ca0821e4120a826f2243794d54b/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:13.854314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=69ea706106c347c59e61a322c7183a7f, user_id=aafe5ca0821e4120a826f2243794d54b, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:13.855783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3e41a-79de-4ff4-83e9-57e7c21953c3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:13.867986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 600/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:57:13 2022] PUT /identity/v3/projects/69ea706106c347c59e61a322c7183a7f/users/aafe5ca0821e4120a826f2243794d54b/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:57:15.812722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188b55-150b-4a6a-805d-a627ab4b0cdd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:15.813285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188b55-150b-4a6a-805d-a627ab4b0cdd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:15.813285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d188b55-150b-4a6a-805d-a627ab4b0cdd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:15.814642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 593/1193] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:57:15.820106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:15.820498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:15.820648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:15.838186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:57:15.839351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:57:15.903510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 601/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:57:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:57:15.926832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d188b55-150b-4a6a-805d-a627ab4b0cdd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.021784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:16.029247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d188b55-150b-4a6a-805d-a627ab4b0cdd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.061242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.061862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.062141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.062413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.072804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:57:16.078148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:57:16.079224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:57:16.080132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 594/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:57:16.085356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.147326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:16.191300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4c515-f97d-4a43-bffc-7016ac9ed389 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.223660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.224265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.224549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.224827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.225569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:16.225879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:16.227243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:16.235528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 602/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:57:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:16.241312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.338718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90098-ab24-4c64-bcc7-ee2d79418bed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.371835 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.372699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.373006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.373290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] PATH_INFO: `/v3/users/aafe5ca0821e4120a826f2243794d54b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.374452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=aafe5ca0821e4120a826f2243794d54b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:16.375879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:16.377492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 595/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Jul 27 19:57:16 2022] GET /identity/v3/users/aafe5ca0821e4120a826f2243794d54b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:16.383118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.490238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42e79d-d3bb-483e-82e1-784aceab60c3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.507193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:16.525083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.525701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.526002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.526259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.591327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:16.592704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:16.687232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 603/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:57:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:57:16.693399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.719538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:16.792150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec42940-ba60-433e-92d4-d6d14d3f5fc2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.829377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.830491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.830992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.831337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] PATH_INFO: `/v3/projects/69ea706106c347c59e61a322c7183a7f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.832509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] RBAC: Authorizing `identity:get_project(project_id=69ea706106c347c59e61a322c7183a7f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:16.834782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f25f0-f455-4a35-84ba-232c184a9714 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:16.836468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 596/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Jul 27 19:57:16 2022] GET /identity/v3/projects/69ea706106c347c59e61a322c7183a7f => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:16.843465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:57:16.861164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:16.941413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c6e7e-ab9c-4c3f-829d-7d71b56dae45 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:57:16.974917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:57:16.975519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:57:16.975799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:57:16.976070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:57:16.976842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=69ea706106c347c59e61a322c7183a7f, user.id=aafe5ca0821e4120a826f2243794d54b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:57:16.977176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:57:16.978600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33994c35-3193-4390-a2d0-d45e96795386 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:57:16.984879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 604/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:57:16 2022] GET /identity/v3/role_assignments?user.id=aafe5ca0821e4120a826f2243794d54b&scope.project.id=69ea706106c347c59e61a322c7183a7f&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:57:26.824194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=54 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:57:26.982224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=60 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:12.337447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7827-99f0-4d38-bc6d-d3d1570b1c95 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:12.338019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7827-99f0-4d38-bc6d-d3d1570b1c95 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:12.338019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7827-99f0-4d38-bc6d-d3d1570b1c95 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:12.339611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 597/1201] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:12.347541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:12.347541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:12.347541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:12.367258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:12.368467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:12.452153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 605/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:12.478621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7827-99f0-4d38-bc6d-d3d1570b1c95 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:12.613265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc7827-99f0-4d38-bc6d-d3d1570b1c95 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:12.669895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:12.670796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:12.671095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:12.671468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:12.686195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:12.688268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:12.706292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:12.707377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:12.708927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:12.708927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:12.708927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:12.708927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:12.708927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 598/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:12.715217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:12.849958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d3946-0065-4628-9f2f-7a33695a8b4f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:12.894245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:12.901672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:12.902353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:12.902648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:12.903047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:12.903875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:12.904180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:12.905658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51231-664f-464d-966a-e0a9c5a38844 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:12.924443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 606/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:12.925935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:13.011600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:13.092845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb7a5e-2ccb-410f-9656-d9b5bd09f141 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:13.142555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:13.143314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:13.143775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:13.144126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:13.144991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:13.146482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35bafe5-0dc4-4ee6-84be-49ed53b7e167 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:13.181569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 599/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Jul 27 19:58:12 2022] POST /identity/v3/users => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:15.500372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd00f9-440e-48b8-8827-79a523e6e360 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.500924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd00f9-440e-48b8-8827-79a523e6e360 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.500924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd00f9-440e-48b8-8827-79a523e6e360 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.502573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 607/1206] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:15.508165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.508605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.508708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.530525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:15.531984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:15.612265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 600/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:15.638831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd00f9-440e-48b8-8827-79a523e6e360 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.769954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd00f9-440e-48b8-8827-79a523e6e360 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:15.811972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:15.812752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:15.813183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:15.813525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:15.829698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:15.836221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:15.837275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:15.838961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 608/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:15.844086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:15.892877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:15.969288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec51851-66ec-457d-a332-1e3f7c602c83 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.007629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.008254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.008544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.008844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:16.009593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:16.009872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:16.011307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:16.019678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 601/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:16.025449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:16.081466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:16.141613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87037a-b7b1-4369-bcf7-552f66dbe07b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.180473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.181456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.181806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.182736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:16.194306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:16.195775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:16.211044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:16.212187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:16.213199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:16.213199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:16.213199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:16.213199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:16.213199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 609/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:16.218697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:16.328180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:16.331396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45510574-43b4-495e-9676-e0c5b38a4192 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.371564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.372286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.372647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.373020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:16.373793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:16.374183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:16.375626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:16.391863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 602/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:16.401404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:16.522016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641709dc-16dc-483c-88ff-a3f1bd73e89b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.523866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:16.565574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.566283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.569022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.569022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:16.576988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:16.578664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:16.592690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:16.593931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:16.594993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:16.596163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 610/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:16.599913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:16.726190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4f2d8-0178-4dfa-9bce-dc12e4d080c8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.771743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.772423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.772698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.773000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:16.773780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:16.774075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:16.776532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:16.801994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 603/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:16.803631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:16.925402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d931a4-a1d3-4052-8cd7-eabbe05ba20f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:16.966052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:16.966680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:16.967089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:16.967421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.005620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:17.055103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:17.056653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:17.179454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 611/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:17.185465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:17.207396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:17.309056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01c375-8575-48f2-8946-3bbfb7e0a829 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:17.345992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:17.347114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.347114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.347294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.360924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:17.362999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:17.381008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:17.383546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:17.385004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:17.385004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:17.385004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:17.385004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:17.387041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 604/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:17.392850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:17.460764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:17.509104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de0cbf-5a9b-4692-86a4-6051d572a8ad None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:17.546204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:17.546863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.547104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.547367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.548057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:17.548392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:17.549800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:17.568796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 612/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:17.569932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:17.645975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:17.677859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ef4c-1e4b-4bdd-8551-6a9a1c0ad164 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:17.716824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:17.717524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.717878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.718259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.726566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:17.728271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:17.741361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:17.742578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:17.743688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:17.744865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 605/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:17.750900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:17.877135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babe2b8-9542-4bc6-8d18-0407d421de3a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:17.908240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:17.924563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:17.925381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:17.925695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:17.926047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:17.926897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:17.927206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:17.930016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48cf9-a92d-439b-96d4-ad7bad24fb11 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:17.942377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 613/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:17.949501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:18.080048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b223-a548-4558-8a02-86ccf4986f2d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:18.117311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:18.124457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:18.125109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:18.125392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:18.125668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:18.126485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6d75086318b148b59d70031805acdc9e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:18.126768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:18.128230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd142a18-cf89-4364-8167-fe9d84a42e6b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:18.133875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 606/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:17 2022] GET /identity/v3/role_assignments?user.id=6d75086318b148b59d70031805acdc9e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:20.203723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e574-1f9c-4a32-aa7f-5d7a93f0a204 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.204295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e574-1f9c-4a32-aa7f-5d7a93f0a204 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.204295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e574-1f9c-4a32-aa7f-5d7a93f0a204 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.205932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 614/1220] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:20.211002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.211433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.211565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.230991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:20.232184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:20.305121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 607/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:20.328276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1e574-1f9c-4a32-aa7f-5d7a93f0a204 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:20.436073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1e574-1f9c-4a32-aa7f-5d7a93f0a204 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:20.472501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:20.473162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.473439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.473708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.483716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:20.488553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:20.489656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:20.495737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 615/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:20.496581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:20.600103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ba964-5af3-42b8-8f31-456b7f038b34 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:20.624075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:20.635891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:20.636504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.636780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.637091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.637803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:20.638101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:20.639459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:20.647437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 608/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:20.654306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:20.701011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:20.758676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd2a47-836d-4ff9-8a06-014de97fe2a9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:20.797517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:20.798577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.799073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.799544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.814247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:20.816523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:20.836857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:20.837934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:20.839131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:20.839131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:20.839131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:20.839131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:20.839131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 616/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:20.844532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:20.959401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40073-774c-4d4c-bfc0-36694cc5f644 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:20.997588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:20.998349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:20.998797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:20.999171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:20.999908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:21.000218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.001786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.015919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 609/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:21.021652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.139014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.147777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcac2a5-b5ba-42a2-8d03-69a27d5c05e4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.188056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.188705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.189013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.189295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.197041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.199644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:21.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:21.213957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:21.215199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 617/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:21.219442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.272900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.332804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50eeaa-f05f-4257-8615-b2e206202c68 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.376208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.378852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.379264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.379702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.381494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:21.381882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.383784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.407425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 610/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:21.413374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.528714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1198ad3-c822-425e-85c7-25f581684c41 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.563894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.564558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.564835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.565155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.576654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.578128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:21.591979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:21.593176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:21.594086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:21.594086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:21.594086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:21.594086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:21.594086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 618/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:21.599831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.713877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c0cb-02b8-4032-9aa4-8a90d5440c31 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.716388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.757731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.758395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.758631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.758944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.759895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:21.760843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.762280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.774333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 611/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:21.780128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:21.789459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:21.894335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c2ad7-e23a-4e09-becd-7a726ff61d93 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:21.929834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:21.930542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:21.930741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:21.931010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:21.938552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:21.940029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:21.952182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:21.953219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:21.954319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 619/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:21.959318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.064467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb0de3-b465-4969-8bb3-3475fa0ee199 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.103364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.104155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.104462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.104741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.105575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:22.105914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.107335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80d138-d8b9-48a6-b4eb-c0456001f95b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.127346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 612/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:22.128395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:22.201734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:22.243003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539537-9381-43d4-b468-b0e5db4052e8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:22.281572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:22.282273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:22.282551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:22.282809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6d75086318b148b59d70031805acdc9e/roles/b2ccc24b9d9943cca2230c8f5651807d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:22.298201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=6d75086318b148b59d70031805acdc9e, role_id=b2ccc24b9d9943cca2230c8f5651807d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:22.299690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109b91-6ba6-4ec7-9ec4-98ed274baf17 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:22.313081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 620/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:58:22 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6d75086318b148b59d70031805acdc9e/roles/b2ccc24b9d9943cca2230c8f5651807d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:58:24.555540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770662a8-a268-4296-8a86-cbae129dc585 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:24.556107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770662a8-a268-4296-8a86-cbae129dc585 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:24.556269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770662a8-a268-4296-8a86-cbae129dc585 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:24.558689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 613/1233] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:24.564043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:24.564632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:24.564632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:24.586674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:24.588589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:24.662024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 621/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:24.687676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770662a8-a268-4296-8a86-cbae129dc585 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:24.716929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:24.802512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770662a8-a268-4296-8a86-cbae129dc585 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:24.853574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:24.854850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:24.855226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:24.855598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:24.867694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:24.872772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:24.874058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:24.875265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 614/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Jul 27 19:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:24.881197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:24.941036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:25.011599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885173a-2d1c-439c-af3c-b0a046514710 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.047459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.048171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.048442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.048716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.049765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:25.050157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:25.051680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:25.060426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 622/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Jul 27 19:58:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:25.066417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.164481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:25.184665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f7917-40c1-4cf4-9c7d-51a44106cd79 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.225892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.226534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.226814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.227131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.238425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:25.239854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:25.256874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:25.265055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:25.267002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:25.267002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:25.267002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:25.267002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:25.267002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 615/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:25.267544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.404111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11063629-b32e-4d07-ad92-1f827d2ac201 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.413321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:25.440838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.441396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.441967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.442212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.442897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:25.443216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:25.444566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:25.456786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 623/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:25.463873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.580142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c46d-1301-4080-a49a-65e48fa598cf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.616203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.616846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.617159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.617438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.620672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:25.627189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:25.629068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:25.641252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:25.642438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:25.643368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 616/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:25.648595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.762959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843aeff-b642-4d70-988b-f818d1279572 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.800988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.801857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.802185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.802582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:25.803641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:25.804095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:25.805585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:25.834353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 624/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:25.836012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:25.956757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059912-f320-46a6-836b-7c8a0e36daba None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:25.992101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:25.992722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:25.993162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:25.993396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.060256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.067549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.069031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.186341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 617/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:26.193160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.224805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.311528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae70e5-cd6c-4901-b017-3d2bb8e5cd30 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.348798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.349471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.349751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.350037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.361464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.362922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:26.380841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:26.381954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:26.383044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:26.383044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:26.383044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:26.383044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:26.383044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 625/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:26.390911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.501977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.509200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cedd38-df94-44e8-be3f-aa863df025e5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.547806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.548424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.548770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.549003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.549702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:26.549992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.551407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.564329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 618/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:26.570332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.665203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:26.690226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15258dc-5705-4dc0-ab51-887e4d931fbf None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.725978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.726617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.726938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.727241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.736537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.738417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:26.753071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:26.755506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:26.756387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 626/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:26.761618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:26.881548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590798-9a0c-4df3-b070-28d01ff588b4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:26.926041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:26.926868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:26.927283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:26.927560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:26.928322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:26.928643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:26.930313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843e78e-f0f9-4f2d-bda7-1ee2a72177c0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:26.945061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 619/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:26.952304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:27.065616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4bdc-90e4-49b9-94f1-4a5ebea2ad05 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:27.105416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:27.106586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:27.106981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:27.107366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:27.108321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] RBAC: Adding query filter params (role.id=b2ccc24b9d9943cca2230c8f5651807d, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6d75086318b148b59d70031805acdc9e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:27.108628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:27.110151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65120b38-0e43-48da-9aae-d427fabb73fd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:27.116833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 627/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:26 2022] GET /identity/v3/role_assignments?user.id=6d75086318b148b59d70031805acdc9e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=b2ccc24b9d9943cca2230c8f5651807d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:29.408705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1e53-4a7b-4b53-a478-da94feba2c4a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.409273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1e53-4a7b-4b53-a478-da94feba2c4a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.409432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d1e53-4a7b-4b53-a478-da94feba2c4a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.411198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 620/1247] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:29.417630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.419402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.419402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.425556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:29.440647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:29.442162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:29.522006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 628/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:29.548411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d1e53-4a7b-4b53-a478-da94feba2c4a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:29.569571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:29.671078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d1e53-4a7b-4b53-a478-da94feba2c4a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:29.715726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:29.716406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.716728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.716996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.735658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:29.740714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:29.742216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:29.743468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 621/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:29.749357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:29.879119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5135a40-dbfc-44bb-bbf0-fb44a1e90341 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:29.912625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:29.918680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:29.919439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:29.919727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:29.920027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:29.920756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:29.921160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:29.922731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:29.933841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 629/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:29.939799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.033547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.061165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdcbc05-6354-434a-a8f4-7d73d96874be None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.102282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.102932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.103230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.104057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.114930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.116401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:30.134393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:30.136231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:30.138483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:30.138483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:30.138483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:30.138483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:30.138483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 622/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:30.143228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.264036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71839cb1-056a-44a4-bc3f-8fcd2f259753 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.298606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.299169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.299453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.299731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.300417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:30.300725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.302143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.314875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 630/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:30.321217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.438873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6960c-5d74-44c3-846a-ec89b5448c58 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.468027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.477290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.477939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.478223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.479034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.487208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.488915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:30.502174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:30.503219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:30.504151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 623/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 19:58:30.511332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.577456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.626979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361d2f4-dd75-4be2-ad7d-68d6eddb0956 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.669191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.669998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.670289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.670607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.671372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:30.671683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.673149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:30.693336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 631/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:30.700329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:30.824074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e28755-b65f-49b7-a45e-582fec6978f1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:30.865574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:30.866862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:30.867305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:30.867846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:30.936824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:30.951323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:30.953068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.067097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 624/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:31.073602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.179239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e812e5-a377-4459-bd6b-f3876e4eedbc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.195685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.221719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.222361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.222640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.222914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.235409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.237022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:31.255115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:31.256134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:31.257533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:31.257533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:31.257533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:31.257533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:31.257533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 632/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:31.263251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.361597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.376124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0d566-83ec-4bbe-b7d5-318abc7748b0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.414597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.415442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.415938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.416369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.417305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.417620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.419521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.432812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 625/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:31.438653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.548356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480b2e-c894-4f62-a0ac-400eeda5b88a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.586758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.587505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.587773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.588050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.595583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.597201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:31.608978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:31.611243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:31.612285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 633/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:31.616994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.727494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0cb2-88fa-4cf3-8dfe-29d2c302ec06 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.767005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.767931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.768295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.768658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.769636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.770119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.772134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849680d-7bab-47d1-ac43-03a60d138dcb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.784826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 626/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:31.792274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:31.829305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:31.922920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a694768-7c33-4de9-9ad6-772acd60e4c1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:31.968360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:31.969058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:31.969343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:31.969622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:31.970437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6d75086318b148b59d70031805acdc9e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:31.970746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:31.972239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50dbb60-b44b-4522-8e8b-4abd56f3be9d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:31.981092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 634/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:31 2022] GET /identity/v3/role_assignments?user.id=6d75086318b148b59d70031805acdc9e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:33.991602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094648-bea3-4a40-9e2a-9053f8c02827 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:33.992132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094648-bea3-4a40-9e2a-9053f8c02827 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:33.992218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54094648-bea3-4a40-9e2a-9053f8c02827 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:33.993603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 627/1261] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:33.998781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:33.999204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:33.999337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.017515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:34.018692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:34.083157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 635/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:34.105761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54094648-bea3-4a40-9e2a-9053f8c02827 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.116665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.211047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54094648-bea3-4a40-9e2a-9053f8c02827 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.244197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.244624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.244905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.245209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.255018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:34.260521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.261617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:34.262675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 628/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:34.267746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.286129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.363315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ea404-ba9a-4d19-b974-b4d15f0b46c9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.396782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.397412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.397692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.397966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.398672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:34.398972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.400686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.408368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 636/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:34.413985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.487433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.520894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f507-cc70-4640-b6db-6363abbb3314 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.553622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.554250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.554531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.554804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.565296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.566985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:34.581789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.582820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.583793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:34.583793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:34.583793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:34.583793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:34.583793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 629/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:34.588717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.666049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.686062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aaf1f-564f-4208-8465-e1aa6111118e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.719294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.719905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.720186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.720464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.721181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:34.721489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.722862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.735036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 637/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:34.740673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:34.850001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727347-3e3e-475e-8ab6-a5afd1aa28ea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:34.875735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:34.894030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:34.894680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:34.894964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:34.895270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:34.902747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:34.904201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:34.915165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:34.916935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:34.918095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 630/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:34.922889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.019135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efdace-3e21-495d-91e3-ab1be4337b11 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.051740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.052345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.052623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.052898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.053711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:35.054015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.055439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.064469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.073872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 638/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:35.079508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.177953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90babec6-4f2e-433b-ba67-221968c87c27 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.210186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.210819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.211102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.211379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.222356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.223788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:35.238295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:35.239343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:35.240431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:35.240431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:35.240431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:35.240431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:35.240431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 631/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:35.245243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.341340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5233ef-9e7f-4e04-84cf-3535512a21cb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.373685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.374285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.374561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.374834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.375509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:35.375809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.377190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.387891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 639/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:35.393516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.424574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.492258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5c53-6ccb-4cb8-8ab1-23662abebdc5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.524416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.525104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.525371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.525645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.532055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.533533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:35.545019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:35.546070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:35.547033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 632/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:35.551807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.608284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.649553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8ed38-e40a-4e9f-b2ee-ee4c3a3dd4e5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.682482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.683090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.683372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.683646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.684382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:35.684686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.686136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81324e1-2c09-4b9b-a53b-0ebd1415cdee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.697994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 640/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:35.704050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:35.793249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:35.801899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fae48c-3c08-4ba2-b3d7-abb667c8a112 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:35.839855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:35.840565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:35.840881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:35.841154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] PATH_INFO: `/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6d75086318b148b59d70031805acdc9e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:35.843168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e9eccd156dd4ed69e5b6dad01362098, user_id=6d75086318b148b59d70031805acdc9e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:35.844759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0a29d-457a-4e01-be99-f213044e1f06 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:35.857297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 633/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Jul 27 19:58:35 2022] PUT /identity/v3/projects/9e9eccd156dd4ed69e5b6dad01362098/users/6d75086318b148b59d70031805acdc9e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 19:58:37.835968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461a76-656f-4ed3-8cb8-07b4890aa7a0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:37.836526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461a76-656f-4ed3-8cb8-07b4890aa7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:37.836526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7461a76-656f-4ed3-8cb8-07b4890aa7a0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:37.838245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 641/1274] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:37.843493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:37.843902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:37.844085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:37.862304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:37.863546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:37.928414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 634/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:37.952994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7461a76-656f-4ed3-8cb8-07b4890aa7a0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.050588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7461a76-656f-4ed3-8cb8-07b4890aa7a0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.067630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.084163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.084793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.085105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.085379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.095862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 19:58:38.100660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:38.102134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:38.104264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 642/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Jul 27 19:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:38.108388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.188353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.207423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864ac3-e2bb-46fc-95b2-8523d4aa9ee7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.241182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.241794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.242075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.242353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.243057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:38.243361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.244916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.252684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 635/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:38.258323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.353690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce121-8dd3-4f0d-b28e-bde7eadb014e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.384635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.385270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.385549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.385818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.395733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.397196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:38.411326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:38.412384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:38.413446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:38.413446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:38.413446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:38.413446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:38.413446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 643/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:38.418776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.515401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f72bc4-501c-497d-bb0c-09e39199fb55 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.548462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.549096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.549379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.549654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.550336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:38.550639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.551995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.563262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 636/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:38.569214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.600203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.667306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b527f84-fade-43e7-bb64-0469d074d9d9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.700988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.701832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.702323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.702818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.711195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.713364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 19:58:38.726291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:38.727410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:38.728455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 644/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:38.733435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:38.737137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:38.841723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cc3fc-2779-440a-ae55-da21a1f6073e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:38.886200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:38.886969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:38.887257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:38.887613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:38.888657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:38.889030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:38.890687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:38.908059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 637/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:38.913849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.000124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:39.010050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9440a-2a0e-4b0d-9864-9b86c38e4016 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.041491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.042083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.042361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.042633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.106967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.108409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.194797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:39.205662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 645/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Jul 27 19:58:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 19:58:39.211435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.270480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:39.307520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04260-87e8-4452-82bc-8f009ef59110 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.341277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.341905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.342184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.342459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.352239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.353752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 19:58:39.367838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:39.368957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:39.369917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 19:58:39.369917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 19:58:39.369917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 19:58:39.369917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:39.369917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 638/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 19:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:39.374625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.470630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad6c0-bf37-424b-b5fa-8ad73b13edcb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.502274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.502870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.503145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.503412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.504085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:39.504380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.505769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.516911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 646/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Jul 27 19:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:39.522454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.620666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60018408-a5c4-498f-8eb9-f1738fb05aaa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.633451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:39.653313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.653935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.654216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.654491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.660852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.662304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 19:58:39.673344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:39.674456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 19:58:39.675379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 639/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Jul 27 19:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:39.680255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.729297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:39.778664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b5665-cfdd-404a-99e6-20d441aa4b7d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.810589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.811202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.811485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.811760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.812499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:39.812806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.814264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59769669-5800-4ad2-bea5-4ed68cc4cb74 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.826645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 647/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Jul 27 19:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:39.832723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:39.927474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a26cd-12da-4032-bd31-5c5857531c2d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:39.959973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:39.960565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:39.960839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:39.961138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:39.961915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] RBAC: Adding query filter params (role.id=656ff46e1a044f1a8f146570c85d81a0, scope.project.id=9e9eccd156dd4ed69e5b6dad01362098, user.id=6d75086318b148b59d70031805acdc9e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:39.962221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:39.963772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094defa-39e8-444e-bd84-b7d6d51c069b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:39.969314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 640/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Jul 27 19:58:39 2022] GET /identity/v3/role_assignments?user.id=6d75086318b148b59d70031805acdc9e&scope.project.id=9e9eccd156dd4ed69e5b6dad01362098&role.id=656ff46e1a044f1a8f146570c85d81a0 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:41.991577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb1ec-01dd-4d85-9ed7-ef48907c1469 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:41.992167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb1ec-01dd-4d85-9ed7-ef48907c1469 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:41.992167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94deb1ec-01dd-4d85-9ed7-ef48907c1469 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:41.994040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 648/1288] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:41.999596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.000049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.000149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:42.018678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:42.019865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:42.052211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:42.090815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 641/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:42.112044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deb1ec-01dd-4d85-9ed7-ef48907c1469 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:42.210843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94deb1ec-01dd-4d85-9ed7-ef48907c1469 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:42.221034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:42.244875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:42.245465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.245743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.246014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:42.246627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:42.247994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:42.261219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:42.262420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:42.263501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 649/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:42.268351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:42.363424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e102c49-bf50-4fa0-8763-fafd302b91cd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:42.395667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:42.396270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.396548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.396818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:42.397510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:42.397811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:42.399158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e7d70-867e-4c12-ab8b-4a300570c657 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:42.403901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 642/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:42.409752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:42.506214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845f7cd-5aee-4174-9fef-29dc966b2ef7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:42.537835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:42.538433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:42.538716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:42.538990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:42.539655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:42.539956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:42.541992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7458ec-e616-4851-b3eb-dfa50720e6c5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:42.546266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 650/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 19:58:44.553854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3d48-f7dc-49c4-b216-413fb85762c8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:44.554404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3d48-f7dc-49c4-b216-413fb85762c8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:44.554404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3d48-f7dc-49c4-b216-413fb85762c8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:44.555727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 643/1293] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:44.560983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6412a-2721-4552-9155-c105eb0420ae None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:44.561372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6412a-2721-4552-9155-c105eb0420ae None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:44.561574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6412a-2721-4552-9155-c105eb0420ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:44.580030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1de6412a-2721-4552-9155-c105eb0420ae None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:44.581247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1de6412a-2721-4552-9155-c105eb0420ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:44.610123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:44.646200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 651/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:44.668738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3d48-f7dc-49c4-b216-413fb85762c8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:44.699189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:44.766121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3d48-f7dc-49c4-b216-413fb85762c8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:44.801044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:44.801652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:44.801929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:44.802202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:44.802890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:44.804300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3f58a-adf2-48f1-a92f-f9f1b69bfbec None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:44.822304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 644/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Jul 27 19:58:44 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:46.797029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faec058-0b38-4826-977b-103d6c64e3d7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:46.797583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faec058-0b38-4826-977b-103d6c64e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:46.797583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faec058-0b38-4826-977b-103d6c64e3d7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:46.798945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 652/1296] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:46.803932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:46.804314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:46.804494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:46.827597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:46.828940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:46.900277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 645/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:46.922867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faec058-0b38-4826-977b-103d6c64e3d7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:47.021574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faec058-0b38-4826-977b-103d6c64e3d7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:47.054695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:47.055269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:47.055550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:47.056340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:47.057214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:47.058597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:47.067517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:47.068682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:47.069664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 653/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:47.075384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:47.093317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:47.172583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f09c24-c8f5-401c-8b3d-3d13a84b7f84 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:47.205727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:47.206341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:47.206624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:47.206901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:47.207577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:47.207887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:47.209283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:47.215587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 646/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:47.221689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:47.252639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:47.319280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109784f4-dd33-4bac-bfe0-d8e27bd8d990 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:47.351630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:47.352228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:47.352510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:47.353369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:47.354129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] RBAC: Adding query filter params (interface=public, service_id=709140abb8a24d618ea0d46268cfeed3, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:47.354434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:47.355798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689aad0-3ccc-44ac-ac6a-54da3eca499a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:47.360639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 654/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Jul 27 19:58:47 2022] GET /identity/v3/endpoints?service_id=709140abb8a24d618ea0d46268cfeed3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:47.366301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:47.444601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:47.463989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51cee2-c72a-44f3-86a3-23b100d5925a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:47.496222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:47.496830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:47.497214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:47.497492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:47.498135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:47.499499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170f16d-0b23-44f6-8571-447a75dfb05a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:47.506118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 647/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Jul 27 19:58:47 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 19:58:49.484268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954a99e-0c5d-45f4-978f-9b936c8c0387 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:49.484812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954a99e-0c5d-45f4-978f-9b936c8c0387 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:49.484812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954a99e-0c5d-45f4-978f-9b936c8c0387 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:49.486188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 655/1302] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:58:49.491439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:49.491839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:49.491988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:49.510575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:58:49.511749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:58:49.576447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 648/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:58:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 19:58:49.598885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954a99e-0c5d-45f4-978f-9b936c8c0387 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:49.695652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:49.703181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954a99e-0c5d-45f4-978f-9b936c8c0387 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:49.735705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:49.736280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:49.736561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:49.736837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:49.737488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:49.738856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 19:58:49.748372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 19:58:49.749508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jul 27 19:58:49.750368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 656/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Jul 27 19:58:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 19:58:49.755536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:49.832121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:58:49.859177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81035-5aea-42fd-a170-87f766004f52 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:49.897439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:49.898052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:49.898329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:49.898596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:49.899259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 19:58:49.899587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:49.900939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab4378-bbf4-4cff-a926-efa9270351f4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:49.905955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 649/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Jul 27 19:58:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 19:58:49.911846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:58:50.013595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a03c8-eaf4-41eb-9599-c2cff9f45635 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:58:50.047777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:58:50.048398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:58:50.048681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:58:50.048957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:58:50.049712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:58:50.051079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b75626-1605-4eba-af83-50022fd4f256 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:58:50.071325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 657/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 19:58:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 19:58:59.902977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=30 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:00.067015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:52.817987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2bee-2b1d-4adf-a65b-f7443cb622a8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:52.819054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2bee-2b1d-4adf-a65b-f7443cb622a8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:52.819715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a2bee-2b1d-4adf-a65b-f7443cb622a8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:52.821925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 650/1307] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:59:52.827928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb64c-4b0d-4f5a-b00c-e9d42901e8e0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:52.828450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb64c-4b0d-4f5a-b00c-e9d42901e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:52.828450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fb64c-4b0d-4f5a-b00c-e9d42901e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:52.910918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-574fb64c-4b0d-4f5a-b00c-e9d42901e8e0 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:59:52.913167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-574fb64c-4b0d-4f5a-b00c-e9d42901e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:53.119493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 658/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 19:59:53.165051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:53.166324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:53.167368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:53.172045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 651/1309] 10.222.0.65 () {54 vars in 965 bytes} [Wed Jul 27 19:59:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:59:53.178878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427de93-3be5-4e7a-a5c1-594a11f14823 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:53.179245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427de93-3be5-4e7a-a5c1-594a11f14823 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:53.179532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427de93-3be5-4e7a-a5c1-594a11f14823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:53.296379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c427de93-3be5-4e7a-a5c1-594a11f14823 None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:59:53.297651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c427de93-3be5-4e7a-a5c1-594a11f14823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:53.320339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:53.382335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 659/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Jul 27 19:59:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 19:59:53.391170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:59:53.392932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:53.511813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315f60a-059e-443a-b735-fcce93c4e888 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:59:53.592625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:59:53.593184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:53.593457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:53.593725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:53.595321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:53.650969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:53.707799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:59:53.709259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834151fe-32f6-443a-9c04-5cc8d0894d3a service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:59:53.728256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 652/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 19:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 19:59:56.040435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:56.040997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:56.040997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:56.042400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 660/1312] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:59:56.047595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cf263-83da-4c91-9e4a-ffac080cce74 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:56.048004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cf263-83da-4c91-9e4a-ffac080cce74 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:56.048156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cf263-83da-4c91-9e4a-ffac080cce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:56.141807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ec3cf263-83da-4c91-9e4a-ffac080cce74 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:59:56.143068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ec3cf263-83da-4c91-9e4a-ffac080cce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:56.247768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 653/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 19:59:56.286369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:59:56.287987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:56.390827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6dce7-23b4-4325-8c2c-822738769efb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:59:56.464812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:56.467316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:59:56.468016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:56.468541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:56.469077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:56.470967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:56.584390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:59:56.586470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777def3-329f-4805-94ba-78167ac81106 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:59:56.604940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 661/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 19:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 19:59:58.970735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fefd0-566e-4430-b039-ad92f554f33a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:58.971312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fefd0-566e-4430-b039-ad92f554f33a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:58.971312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fefd0-566e-4430-b039-ad92f554f33a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:58.972637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 654/1315] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 19:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:59:58.977824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cc16-7de0-484a-93c2-23799ed8bc56 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:58.978252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cc16-7de0-484a-93c2-23799ed8bc56 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:58.978378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cc16-7de0-484a-93c2-23799ed8bc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:59.043426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1e34cc16-7de0-484a-93c2-23799ed8bc56 None None] MFA Rules not processed for user `657440e9843c4f8097ef973d0719da30`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:59:59.044658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e34cc16-7de0-484a-93c2-23799ed8bc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:59.101482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:59.151429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 662/1316] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 19:59:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 19:59:59.202854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:59.203139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:59.203416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:59.204761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 655/1317] 10.222.0.65 () {54 vars in 965 bytes} [Wed Jul 27 19:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 19:59:59.211215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077510-8095-437a-b0d7-de78ed92ebfe None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:59.211537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077510-8095-437a-b0d7-de78ed92ebfe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:59.211817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077510-8095-437a-b0d7-de78ed92ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:59.288430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bd077510-8095-437a-b0d7-de78ed92ebfe None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 19:59:59.289585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd077510-8095-437a-b0d7-de78ed92ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:59.367490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 663/1318] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Jul 27 19:59:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 19:59:59.376246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 19:59:59.377814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:59.387787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:59.483335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c3c19-4b5f-433c-a539-b9836b9cd7a3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 19:59:59.553859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 19:59:59.554391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 19:59:59.554667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 19:59:59.554939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 19:59:59.556546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 19:59:59.617736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 19:59:59.663678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 19:59:59.665141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e1a45-c137-4ad8-9d15-570e82f228fc service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 19:59:59.682651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 656/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 19:59:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:02.676946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:02.677511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:02.677511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:02.678995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 664/1320] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:02.684164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9ebc7-3555-4715-af7a-e0fa288532c1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:02.684562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9ebc7-3555-4715-af7a-e0fa288532c1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:02.684727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9ebc7-3555-4715-af7a-e0fa288532c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:02.749432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dbe9ebc7-3555-4715-af7a-e0fa288532c1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:02.750659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dbe9ebc7-3555-4715-af7a-e0fa288532c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:02.848256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 657/1321] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:02.887789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:02.889396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:02.938243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:02.992649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19ea8-0626-4b44-8662-84c91a87df19 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:03.064250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:03.064860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:03.065176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:03.065457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:03.067357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:03.163852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:03.176851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:03.178925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d790-f82d-4399-97b3-efc011eb49f7 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:03.197216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 665/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:05.877493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:05.878055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:05.878055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:05.879576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 658/1323] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:05.885034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f121-96bb-4f3d-81ac-50148f66a2fa None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:05.885430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f121-96bb-4f3d-81ac-50148f66a2fa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:05.885582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f121-96bb-4f3d-81ac-50148f66a2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:05.949293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0483f121-96bb-4f3d-81ac-50148f66a2fa None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:05.950485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0483f121-96bb-4f3d-81ac-50148f66a2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:06.051976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 666/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:06.088947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:06.090584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:06.113561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:06.206775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab22fc-7626-4429-8a6c-a954a852c3f2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:06.284031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:06.284665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:06.284944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:06.285246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:06.287006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:06.363010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:06.401404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:06.402871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9082010-a37c-4108-bea9-b1c183dc5b0e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:06.420828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 659/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:00:08.471457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:08.472018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:08.472018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:08.473396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 667/1326] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:08.479187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1f3d5-c81e-48e5-9706-9d977e5cc43e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:08.479625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1f3d5-c81e-48e5-9706-9d977e5cc43e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:08.479718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1f3d5-c81e-48e5-9706-9d977e5cc43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:08.545896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ade1f3d5-c81e-48e5-9706-9d977e5cc43e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:08.547114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ade1f3d5-c81e-48e5-9706-9d977e5cc43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:08.645210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 660/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:08.682103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:08.683654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:08.744867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:08.789014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699dfcb-ada9-4edf-82bf-f30adce236ff None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:08.868740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:08.869377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:08.869657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:08.869926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:08.871538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:08.976673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:08.979512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:08.981116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6a42d-f0fb-448a-bdca-03ddae286b00 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:08.999795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 668/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:11.088520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:11.089041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:11.089159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:11.090614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 661/1329] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:11.095963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9fd6f-4382-453e-a690-a1bc91a0c91e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:11.096381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9fd6f-4382-453e-a690-a1bc91a0c91e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:11.096526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9fd6f-4382-453e-a690-a1bc91a0c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:11.164840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-54f9fd6f-4382-453e-a690-a1bc91a0c91e None None] MFA Rules not processed for user `657440e9843c4f8097ef973d0719da30`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:11.166094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54f9fd6f-4382-453e-a690-a1bc91a0c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:11.265074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 669/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:11.303393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:11.304945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:11.336677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:11.411875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258176f-6cd4-4a7e-b9b1-568d4c712428 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:11.493579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:11.494207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:11.494486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:11.494764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:11.496447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:11.572733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:11.604829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:11.606241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f0486-e542-4dae-ba4a-c997c56d30c1 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:11.624812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 662/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:13.893332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:13.893924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:13.893924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:13.895229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 670/1332] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:13.900366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec09965-9f95-4e6e-b0fd-c32327be49a3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:13.900773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec09965-9f95-4e6e-b0fd-c32327be49a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:13.901046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec09965-9f95-4e6e-b0fd-c32327be49a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:13.967164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dec09965-9f95-4e6e-b0fd-c32327be49a3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:13.968384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dec09965-9f95-4e6e-b0fd-c32327be49a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:14.063433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 663/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:14.100931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:14.102499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:14.171477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:14.206040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690188e3-feff-4633-8da4-74060e41bec3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:14.282421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:14.283030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:14.283308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:14.283582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:14.285225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:14.392896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:14.394347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ed508-4c07-4237-b1de-c674112c3de3 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:14.405080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:14.414031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 671/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:00:16.482178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:16.482736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:16.482736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:16.484124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 664/1335] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:16.489196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedcde-43df-4194-ae94-f39d60d6025e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:16.489587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedcde-43df-4194-ae94-f39d60d6025e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:16.489852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedcde-43df-4194-ae94-f39d60d6025e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:16.556051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ebcedcde-43df-4194-ae94-f39d60d6025e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:16.557292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ebcedcde-43df-4194-ae94-f39d60d6025e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:16.651945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 672/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:16.693737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:16.695260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:16.735018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:16.796880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b2f85-793b-4fc9-ab04-aa7185ce245f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:16.868392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:16.869017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:16.869296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:16.869563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:16.871168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:16.955016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:16.980680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:16.982150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6d796-0189-479f-9f86-820ccbcda234 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:17.000074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 665/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:19.350687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:19.351210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:19.351210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:19.352657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 673/1338] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:19.358343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbaf97-77f2-405f-928a-457753ba265d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:19.358742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbaf97-77f2-405f-928a-457753ba265d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:19.358899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbaf97-77f2-405f-928a-457753ba265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:19.423476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-82bbaf97-77f2-405f-928a-457753ba265d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:19.424694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82bbaf97-77f2-405f-928a-457753ba265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:19.519445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 666/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:19.556440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:19.558566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:19.635734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:19.664366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99bb52-63c6-4d5b-9a78-c58db0a78cfe None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:19.735885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:19.736503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:19.736785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:19.737091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:19.738719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:19.855411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:19.856883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2e10-e07e-488a-80a1-ef0df0ec384c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:19.874841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 674/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:23.872216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:23.872740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:23.872740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:23.874092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 667/1341] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:23.879280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bf37-fec1-4964-8672-eb0bf425e974 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:23.879752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bf37-fec1-4964-8672-eb0bf425e974 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:23.879977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146bf37-fec1-4964-8672-eb0bf425e974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:23.885801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:23.949017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9146bf37-fec1-4964-8672-eb0bf425e974 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:23.950220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9146bf37-fec1-4964-8672-eb0bf425e974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:24.046206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 675/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:24.085094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:24.086625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:24.135640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:24.193472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4339f1-432c-4364-b10c-a90ca084e5cd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:24.268068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:24.268686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:24.268979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:24.269262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:24.270899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:24.367386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:24.384976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:24.386484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861aed-6efc-40d9-8764-86ea79326212 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:24.405447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 668/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:27.199237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:27.199790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:27.199790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:27.201278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 676/1344] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:27.206980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3b1e-142f-4202-abe1-937fe87ac393 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:27.207532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3b1e-142f-4202-abe1-937fe87ac393 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:27.207532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3b1e-142f-4202-abe1-937fe87ac393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:27.279004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a6da3b1e-142f-4202-abe1-937fe87ac393 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:27.280239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a6da3b1e-142f-4202-abe1-937fe87ac393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:27.381939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 669/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:27.419124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:27.420779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:27.509425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:27.527728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd30a-58ac-42ea-8505-d35e5e1904f8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:27.603413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:27.604031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:27.604313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:27.604587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:27.606232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:27.717939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:27.719371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f67216-1412-4c81-b69b-f1e5ecd7496b service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:27.736766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 677/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:31.737988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:31.738528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:31.738528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:31.740823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 670/1347] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:31.746028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494311-3168-457b-9fda-63736c089073 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:31.746460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494311-3168-457b-9fda-63736c089073 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:31.746748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7494311-3168-457b-9fda-63736c089073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:31.762637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:31.813872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f7494311-3168-457b-9fda-63736c089073 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:31.815101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f7494311-3168-457b-9fda-63736c089073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:31.916487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 678/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:31.955227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:31.956801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:32.035068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:32.087151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a189d8-0c5c-4cda-abbe-81fbae309d49 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:32.165882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:32.166501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:32.166787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:32.167060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:32.168690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:32.281524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:32.286944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:32.288398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ae092-42ea-4f28-8a5a-7d64bec33109 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:32.307227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 671/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:37.301061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac048-79e3-49a7-8157-72d4944a3822 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:37.301061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac048-79e3-49a7-8157-72d4944a3822 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:37.301061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac048-79e3-49a7-8157-72d4944a3822 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:37.304986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 679/1350] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:37.312309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f585e1-e3e6-4484-a87e-9effad0af7a0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:37.312782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f585e1-e3e6-4484-a87e-9effad0af7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:37.312782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f585e1-e3e6-4484-a87e-9effad0af7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:37.408269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a2f585e1-e3e6-4484-a87e-9effad0af7a0 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:37.409647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a2f585e1-e3e6-4484-a87e-9effad0af7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:37.537506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 672/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:41.740228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf80d5-b3f3-44ff-ade2-978e75a13d59 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:41.740738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf80d5-b3f3-44ff-ade2-978e75a13d59 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:41.740850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf80d5-b3f3-44ff-ade2-978e75a13d59 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:41.742158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 680/1352] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:41.747244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf012e1-30e9-4b92-be4a-7f0b1e9a318d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:41.747658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf012e1-30e9-4b92-be4a-7f0b1e9a318d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:41.747964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf012e1-30e9-4b92-be4a-7f0b1e9a318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:41.806078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:41.831425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-abf012e1-30e9-4b92-be4a-7f0b1e9a318d None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:41.833173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-abf012e1-30e9-4b92-be4a-7f0b1e9a318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:41.914649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=59 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:41.951660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 673/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:43.220569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676be0-b691-4f50-a2a6-6e6ee9b24367 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:43.221104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676be0-b691-4f50-a2a6-6e6ee9b24367 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:43.221219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2676be0-b691-4f50-a2a6-6e6ee9b24367 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:43.222513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 681/1354] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:43.228504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c0c2-8839-4cd5-812c-7a96b8eb23ae None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:43.228918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c0c2-8839-4cd5-812c-7a96b8eb23ae None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:43.229107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0c0c2-8839-4cd5-812c-7a96b8eb23ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:43.314995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-69e0c0c2-8839-4cd5-812c-7a96b8eb23ae None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:43.316454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-69e0c0c2-8839-4cd5-812c-7a96b8eb23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:43.359655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:43.424412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 674/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:00:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:00:46.825397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac8a9-b3bb-4d3a-aee6-dbd3a01f809f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:46.826246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac8a9-b3bb-4d3a-aee6-dbd3a01f809f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:46.826677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac8a9-b3bb-4d3a-aee6-dbd3a01f809f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:46.828185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 682/1356] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:00:46.835405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff067-0e06-4383-8bc9-bee3ab428f37 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:46.835894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff067-0e06-4383-8bc9-bee3ab428f37 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:46.835994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff067-0e06-4383-8bc9-bee3ab428f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:46.924473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-988ff067-0e06-4383-8bc9-bee3ab428f37 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:46.925844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-988ff067-0e06-4383-8bc9-bee3ab428f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:47.043506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 675/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:00:47.102947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:47.103396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:47.103833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:47.105610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 683/1358] 10.222.0.65 () {54 vars in 962 bytes} [Wed Jul 27 20:00:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:47.113004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfc345-6ebc-4ea0-a511-9ffbf9078073 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:47.113298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfc345-6ebc-4ea0-a511-9ffbf9078073 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:47.113579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfc345-6ebc-4ea0-a511-9ffbf9078073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:47.134087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:47.198118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-82bfc345-6ebc-4ea0-a511-9ffbf9078073 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:47.199343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82bfc345-6ebc-4ea0-a511-9ffbf9078073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:47.322464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 676/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Jul 27 20:00:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:47.330309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:47.331876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:47.485975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1ebc-68dd-450b-8b73-4499fb34fb49 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:47.575058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:47.619966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:47.620518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:47.620789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:47.621096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:47.622713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:47.722069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:47.723718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e6a3b-8103-4899-93d5-08e341995010 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:47.744913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 684/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:00:49.997820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:49.998221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:49.998445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:50.000996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 677/1361] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:50.005531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b899-32db-4ca7-b9e2-d064994b5a3e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:50.006592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b899-32db-4ca7-b9e2-d064994b5a3e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:50.007075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b899-32db-4ca7-b9e2-d064994b5a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:50.093084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9298b899-32db-4ca7-b9e2-d064994b5a3e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:50.094876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9298b899-32db-4ca7-b9e2-d064994b5a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:50.135411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:50.204267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 685/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:50.250870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:50.252389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:50.326897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:50.437817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8db30d-a846-4322-a424-104641d69cfd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:50.569289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:50.569920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:50.570202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:50.570479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:50.572120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:50.596257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:50.668712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:50.670367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6967-2f7b-4ebe-a28d-a85aabcb9eb9 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:50.690215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 678/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:53.047798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b290-2784-4fa1-9d42-75cbe604a884 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:53.048318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b290-2784-4fa1-9d42-75cbe604a884 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:53.048920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b290-2784-4fa1-9d42-75cbe604a884 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:53.051216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 686/1364] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:53.056947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90719d3-c8a6-4588-b5b4-55c5f2183296 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:53.057445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90719d3-c8a6-4588-b5b4-55c5f2183296 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:53.057813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90719d3-c8a6-4588-b5b4-55c5f2183296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:53.133325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a90719d3-c8a6-4588-b5b4-55c5f2183296 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:53.134571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a90719d3-c8a6-4588-b5b4-55c5f2183296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:53.234228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:53.257371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 679/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:00:53.287825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:53.288156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:53.288456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:53.290485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 687/1366] 10.222.0.65 () {54 vars in 962 bytes} [Wed Jul 27 20:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:53.295715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82113e-9009-4581-be9e-ce9f107dcce2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:53.296013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82113e-9009-4581-be9e-ce9f107dcce2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:53.296296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82113e-9009-4581-be9e-ce9f107dcce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:53.378005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7d82113e-9009-4581-be9e-ce9f107dcce2 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:53.379173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7d82113e-9009-4581-be9e-ce9f107dcce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:53.505129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 680/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Jul 27 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:53.512892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:53.514598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:53.682058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fd276-99fc-42a6-9803-ba213adbb6e8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:53.704122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:53.807819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:53.808438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:53.808716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:53.809005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:53.810735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:53.907743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:53.909592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefcf7f-325d-49a2-bf63-6250b9e57bdd service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:53.930104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 688/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:00:54.889016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1755f-a05d-459a-aa7c-7f86ce4692d0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:54.889564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1755f-a05d-459a-aa7c-7f86ce4692d0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:54.889664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1755f-a05d-459a-aa7c-7f86ce4692d0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:54.891289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 681/1369] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:54.897678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffe304-42c9-4b9b-8221-cff753292eef None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:54.898192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffe304-42c9-4b9b-8221-cff753292eef None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:54.898276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffe304-42c9-4b9b-8221-cff753292eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:54.932762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:54.971032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6cffe304-42c9-4b9b-8221-cff753292eef None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:54.972236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6cffe304-42c9-4b9b-8221-cff753292eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:55.085328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 689/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:00:55.426719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:55.426719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:55.426719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:55.429993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 682/1371] 10.222.0.65 () {54 vars in 964 bytes} [Wed Jul 27 20:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:55.432840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91f86-498c-4787-b6c5-2e0e4a841ba6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:55.433173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91f86-498c-4787-b6c5-2e0e4a841ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:55.433455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d91f86-498c-4787-b6c5-2e0e4a841ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:55.516020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d5d91f86-498c-4787-b6c5-2e0e4a841ba6 None None] MFA Rules not processed for user `e5c57fe9a7f141458ac3d01bfa045695`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:55.517407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5d91f86-498c-4787-b6c5-2e0e4a841ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:55.566580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:55.599874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 690/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Jul 27 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:55.607235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:55.608737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:55.662205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:55.723538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fe4ef-6ed1-4051-825e-b688260da9b8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:55.801581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:55.802195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:55.802472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:55.802743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:55.804367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:55.907659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:55.927956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:55.929416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d673d9c-fc97-42ef-80e7-716e61e4dcc2 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:55.948689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 683/1373] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:56.073839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:56.074277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:56.074870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:56.077156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 691/1374] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:56.083430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c11-1c68-495b-acec-eb43f088e23b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:56.083728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c11-1c68-495b-acec-eb43f088e23b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:56.083965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a00c11-1c68-495b-acec-eb43f088e23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:56.151755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-87a00c11-1c68-495b-acec-eb43f088e23b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:56.153001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-87a00c11-1c68-495b-acec-eb43f088e23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:56.252380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 684/1375] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:00:56.297719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:56.299314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:56.455931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55440-12ab-438b-b204-b6d36dc6257e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:56.506505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:56.569317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:56.569939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:56.570220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:56.570493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:56.572119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:56.666423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:56.668059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe9c43-eb92-4fd8-b783-0fcdb3843216 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:56.687712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 692/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:00:58.732370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:58.732922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:58.733065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:58.734581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 685/1377] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:00:58.739711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767fcc-2649-49c4-aace-871d5962a3f1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:58.740315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767fcc-2649-49c4-aace-871d5962a3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:58.740537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767fcc-2649-49c4-aace-871d5962a3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:58.818511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-84767fcc-2649-49c4-aace-871d5962a3f1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:00:58.819727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-84767fcc-2649-49c4-aace-871d5962a3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:58.826675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:58.927083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 693/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:00:58.974351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:00:58.975860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:59.057927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:59.145611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fc6db-6f9a-498e-92e9-c298e6c9fed9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:00:59.267906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:00:59.268407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:00:59.268680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:00:59.268976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:00:59.271565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:00:59.290504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:00:59.384103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:00:59.385752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7482c-6b6d-48ef-a269-aaecaec07835 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:00:59.403712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 686/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:01:01.492895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:01.493490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:01.493490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:01.495074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 694/1380] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:01.500639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1ef7a-5842-452b-b58c-8243ced5a673 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:01.501197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1ef7a-5842-452b-b58c-8243ced5a673 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:01.501197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1ef7a-5842-452b-b58c-8243ced5a673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:01.579509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-95d1ef7a-5842-452b-b58c-8243ced5a673 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:01.580632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-95d1ef7a-5842-452b-b58c-8243ced5a673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:01.680888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:01.693496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 687/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:01.741395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:01.742933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:01.901556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:01.912925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8398e-a224-45e5-a5a1-c7c962e75a40 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:02.034607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:02.036163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:02.036670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:02.036990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:02.038632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:02.146353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:02.148011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f32518-c9dd-4b73-9e16-e131517cd2ef service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:02.158900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:02.175392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 695/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:04.293512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:04.294043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:04.294137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:04.295502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 688/1383] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:04.302604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e9004-2f10-458c-a5d0-0ff4f77ebe8b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:04.303044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e9004-2f10-458c-a5d0-0ff4f77ebe8b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:04.303261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e9004-2f10-458c-a5d0-0ff4f77ebe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:04.374529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f61e9004-2f10-458c-a5d0-0ff4f77ebe8b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:04.375976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f61e9004-2f10-458c-a5d0-0ff4f77ebe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:04.477576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 696/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:04.522214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:04.524184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:04.724110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e7878-6358-4317-a9f6-06d9a661f05a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:04.805660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:04.856015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:04.856659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:04.856953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:04.857255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:04.859263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:04.962786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:04.964418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acde262-081c-4acd-8a2e-998eb39b583e service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:04.982617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 689/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:07.076488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:07.076947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:07.077093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:07.078465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 697/1386] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:07.085114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a69cc-338b-4602-8bf4-0d103b4e0558 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:07.085554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a69cc-338b-4602-8bf4-0d103b4e0558 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:07.085678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a69cc-338b-4602-8bf4-0d103b4e0558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:07.153771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-119a69cc-338b-4602-8bf4-0d103b4e0558 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:07.155016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-119a69cc-338b-4602-8bf4-0d103b4e0558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:07.196446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:07.261497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 690/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:07.307876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:07.309886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:07.392934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:07.479105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054713c8-fdf0-4e7a-b30d-eebc8dba0916 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:07.599811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:07.600456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:07.600774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:07.601113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:07.602748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:07.645198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:07.709584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:07.711537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d8180-0754-417d-a32f-aa9d9e64432a service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:07.734997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 698/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:09.926436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:09.927037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:09.927037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:09.928523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 691/1389] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:09.933578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3aff-b3ff-4bf2-8936-a343dff8c581 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:09.933972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3aff-b3ff-4bf2-8936-a343dff8c581 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:09.934257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3aff-b3ff-4bf2-8936-a343dff8c581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:10.001460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-18aa3aff-b3ff-4bf2-8936-a343dff8c581 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:10.002685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-18aa3aff-b3ff-4bf2-8936-a343dff8c581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:10.107020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 699/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:10.152885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:10.154559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:10.305331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed87ced-88a2-431f-8b8b-b84f60272cbc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:10.321583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:10.418777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:10.419396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:10.419731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:10.420016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:10.421675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:10.511038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:10.512648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c40eff-bb33-41fe-9f00-64e0247874dd service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:10.531465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 692/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:11.039585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26b8-2750-4c71-9005-861a6462cf2d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:11.040150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26b8-2750-4c71-9005-861a6462cf2d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:11.040150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26b8-2750-4c71-9005-861a6462cf2d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:11.041560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 700/1392] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:11.046727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335a9-c474-4f34-b1ad-e959aebc1cbd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:11.047129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335a9-c474-4f34-b1ad-e959aebc1cbd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:11.047278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43335a9-c474-4f34-b1ad-e959aebc1cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:11.074069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:11.116783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a43335a9-c474-4f34-b1ad-e959aebc1cbd None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:11.118039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a43335a9-c474-4f34-b1ad-e959aebc1cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:11.217808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 693/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:11.278319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:11.278929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:11.279504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:11.282054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 701/1394] 10.222.0.65 () {54 vars in 966 bytes} [Wed Jul 27 20:01:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:11.288036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875edb7-2200-412c-9ed9-8a3c7b6aa172 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:11.288322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875edb7-2200-412c-9ed9-8a3c7b6aa172 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:11.288623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875edb7-2200-412c-9ed9-8a3c7b6aa172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:11.363469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6875edb7-2200-412c-9ed9-8a3c7b6aa172 None None] MFA Rules not processed for user `d4256769bf5b4e81a15b985039cdbd80`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:11.364584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6875edb7-2200-412c-9ed9-8a3c7b6aa172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:11.403928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:11.439850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 694/1395] 10.222.0.65 () {58 vars in 1063 bytes} [Wed Jul 27 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:11.447544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:11.449072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:11.462345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:11.552838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5e020-3776-41e5-a573-b4d5245aa651 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:11.625837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:11.626377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:11.626663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:11.626939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:11.628621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:11.698536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:11.740710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:11.742269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d120e-f6f2-48bd-94d5-4d7c6fd8ee33 service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:11.760830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 702/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:12.512696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc629-bf5f-402d-8781-9956d0329e1b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:12.513200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc629-bf5f-402d-8781-9956d0329e1b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:12.513427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc629-bf5f-402d-8781-9956d0329e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:12.580870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269b3ef-4185-4bfe-97e4-e6a984694593 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:12.581520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269b3ef-4185-4bfe-97e4-e6a984694593 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:12.581667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269b3ef-4185-4bfe-97e4-e6a984694593 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:12.582825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-052dc629-bf5f-402d-8781-9956d0329e1b None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:12.582956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 703/1397] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:01:12.584125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-052dc629-bf5f-402d-8781-9956d0329e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:12.588081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:12.588368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:12.588651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:12.654587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:12.655813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:12.762870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 704/1398] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:01:12.795480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 695/1399] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:12.811028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:12.812262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:12.841452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070587f7-075d-4841-831c-bc304f7c8ed9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:12.841741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070587f7-075d-4841-831c-bc304f7c8ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:12.842202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070587f7-075d-4841-831c-bc304f7c8ed9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:12.842378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:12.843752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 696/1400] 10.222.0.65 () {54 vars in 943 bytes} [Wed Jul 27 20:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:01:12.851308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:12.851661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:12.852005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:12.920684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:12.929692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] MFA Rules not processed for user `bc49a64077c94d1194086179c518209e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:12.930903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:12.931726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8988-2221-432e-8fd3-588064ff318a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:13.041356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 697/1401] 10.222.0.65 () {58 vars in 1040 bytes} [Wed Jul 27 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:01:13.050756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:13.053907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:13.054729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:13.055018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:13.055472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:13.057180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:13.060259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ceb0-e8b5-49fe-824a-2afaf8c27ad3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:13.076655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:13.149828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:13.151422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c0504-5aea-4458-9291-ffa8eaaf042a service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:13.170644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 705/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:13.173568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:13.174189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:13.174469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:13.174743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:13.176659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:13.179151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:13.267573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:13.269263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ffe75-1812-436f-bf30-fb7586e69b27 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:13.278452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 698/1403] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 20:01:15.235331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:15.235955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:15.235955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:15.237593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 706/1404] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:15.242678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7664cc-6815-4f71-ac7f-656a57c1ae84 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:15.243128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7664cc-6815-4f71-ac7f-656a57c1ae84 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:15.243231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7664cc-6815-4f71-ac7f-656a57c1ae84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:15.314686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8e7664cc-6815-4f71-ac7f-656a57c1ae84 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:15.316816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e7664cc-6815-4f71-ac7f-656a57c1ae84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:15.418488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:15.433533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 699/1405] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:15.473372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:15.475274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:15.638388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:15.641978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3136aa-d553-4c2f-97c0-d1f357b7f14a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:15.769292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:15.769885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:15.770166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:15.770437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:15.773222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:15.872596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:15.874234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fb950-04b9-4642-958c-acfd0854834a service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:15.901155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 707/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:17.978084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:17.978643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:17.978643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:17.980253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 700/1407] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:17.986835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907ed7-ebef-4167-a6a0-914fae8da67f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:17.987272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907ed7-ebef-4167-a6a0-914fae8da67f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:17.987473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907ed7-ebef-4167-a6a0-914fae8da67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:17.993103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:18.061942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cd907ed7-ebef-4167-a6a0-914fae8da67f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:18.063154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd907ed7-ebef-4167-a6a0-914fae8da67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:18.181174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 708/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:18.226383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:18.227895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:18.396897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227bf308-218c-4788-9f8c-e0c3af42b665 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:18.481628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:18.540420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:18.541113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:18.541400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:18.541680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:18.543708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:18.653344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:18.655370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb916f2-b179-4826-bbe2-5477a70a9c19 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:18.678789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 701/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:28.179345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=64 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:28.673666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=64 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:35.751174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb3fa-19c6-4358-bd15-882b536ff3f1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:35.751746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb3fa-19c6-4358-bd15-882b536ff3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:35.751746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbb3fa-19c6-4358-bd15-882b536ff3f1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:35.753285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 709/1410] 10.222.0.65 () {54 vars in 918 bytes} [Wed Jul 27 20:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:35.758408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb2bc-44b9-486d-a2bf-672703aa7c8a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:35.758806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb2bc-44b9-486d-a2bf-672703aa7c8a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:35.758967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fb2bc-44b9-486d-a2bf-672703aa7c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:35.833856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-af5fb2bc-44b9-486d-a2bf-672703aa7c8a None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:35.835064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-af5fb2bc-44b9-486d-a2bf-672703aa7c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:35.939636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 702/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Jul 27 20:01:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:35.954700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f848a-3c2c-4d96-b46d-545c101b7e15 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:35.954987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f848a-3c2c-4d96-b46d-545c101b7e15 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:35.955268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f848a-3c2c-4d96-b46d-545c101b7e15 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:35.956618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 710/1412] 10.222.0.65 () {54 vars in 918 bytes} [Wed Jul 27 20:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:35.961099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2bdf-2255-4cde-855e-304d8202422b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:35.961389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2bdf-2255-4cde-855e-304d8202422b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:35.961664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2bdf-2255-4cde-855e-304d8202422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:36.033076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:36.036386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4e0d2bdf-2255-4cde-855e-304d8202422b None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:36.038291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4e0d2bdf-2255-4cde-855e-304d8202422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:36.153580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 703/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Jul 27 20:01:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:38.637240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:38.637772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:38.637772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:38.639141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:38.644289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0850d-8a65-4b01-bff9-eb9fb5538e04 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:38.645297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0850d-8a65-4b01-bff9-eb9fb5538e04 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:38.645466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0850d-8a65-4b01-bff9-eb9fb5538e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:38.711291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-20a0850d-8a65-4b01-bff9-eb9fb5538e04 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:38.712485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-20a0850d-8a65-4b01-bff9-eb9fb5538e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:38.767087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:38.813099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:38.856449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:38.858096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:39.002246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f0349-8ba3-4916-a4a5-38e2c09f5f90 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:39.074295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:39.119080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:39.119701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:39.119981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:39.120257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:39.121928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:39.209440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:39.211059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79827b6e-27b7-4553-b1ad-8a34be769531 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:39.229574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 712/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:40.133943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:40.134510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:40.134510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:40.135854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:40.140758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132cd6-a8d6-4a0c-a4ff-b7ce92665323 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:40.141209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132cd6-a8d6-4a0c-a4ff-b7ce92665323 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:40.141339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132cd6-a8d6-4a0c-a4ff-b7ce92665323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:40.215883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5d132cd6-a8d6-4a0c-a4ff-b7ce92665323 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:40.217102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5d132cd6-a8d6-4a0c-a4ff-b7ce92665323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:40.244217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:40.313587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:40.485491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:40.487029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:40.632671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b8add-8566-4fcf-97cf-588bb21e2388 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:40.672322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:40.745811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:40.746434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:40.746715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:40.746992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:40.748636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:40.838655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:40.840290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307594d-8e9f-47df-a906-f3fb76e2c02a service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:40.858085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 706/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:41.739680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:41.740452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:41.740733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:41.742245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:41.747180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf12e-8626-4e15-b321-23634dc07474 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:41.747565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf12e-8626-4e15-b321-23634dc07474 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:41.747725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf12e-8626-4e15-b321-23634dc07474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:41.803502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:41.825751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4fdbf12e-8626-4e15-b321-23634dc07474 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:41.826953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4fdbf12e-8626-4e15-b321-23634dc07474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:41.921432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:42.076956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:42.078576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:42.223525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b97a3c-98a5-40e7-8481-1b3e1e3cc077 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:42.235422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:42.335835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:42.336617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:42.337156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:42.337641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:42.339971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:42.431299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:42.432935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec44769-5352-4191-9659-3bc5d7f0b7b5 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:42.451263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 715/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:43.318005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:43.318574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:43.318574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:43.319909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:43.324880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7f660-4553-438b-ad3d-5a195f1c4743 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:43.325364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7f660-4553-438b-ad3d-5a195f1c4743 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:43.325459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7f660-4553-438b-ad3d-5a195f1c4743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:43.344615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:43.400772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9ec7f660-4553-438b-ad3d-5a195f1c4743 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:43.402154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9ec7f660-4553-438b-ad3d-5a195f1c4743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:43.497682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:43.647875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:43.649426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:43.758071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:43.794244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602a0ac-6e20-48d9-9c37-5036e63b737c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:43.904083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:43.904622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:43.904894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:43.905188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:43.906785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:43.981156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:43.994951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:43.996531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222457d3-b4b6-44ec-9918-5d058d514302 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:44.014548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 709/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:44.885305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:44.885874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:44.885874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:44.887261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:44.892120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d11d0-e93b-4926-9958-677d46283470 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:44.892515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d11d0-e93b-4926-9958-677d46283470 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:44.892675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d11d0-e93b-4926-9958-677d46283470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:44.968545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e28d11d0-e93b-4926-9958-677d46283470 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:44.969841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e28d11d0-e93b-4926-9958-677d46283470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:45.065317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:45.215408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:45.216868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:45.299112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:45.360114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28dd6a1-c98b-4f29-b65b-5978217f13a4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:45.477139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:45.477677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:45.477956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:45.478230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:45.479852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:45.528294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:45.571075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:45.572669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c233f78-3ee1-4ca2-b90b-dd99f026a4d2 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:45.590639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 718/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:46.394478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:46.395250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:46.395250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:46.396404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:46.403546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963d65-918e-45a5-b9a1-f7a5af12a816 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:46.404219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963d65-918e-45a5-b9a1-f7a5af12a816 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:46.404219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25963d65-918e-45a5-b9a1-f7a5af12a816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:46.482813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-25963d65-918e-45a5-b9a1-f7a5af12a816 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:46.484036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-25963d65-918e-45a5-b9a1-f7a5af12a816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:46.579981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:46.727901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:46.729379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:46.777371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:46.879037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da1ed5-c8f4-4b92-b631-c7110f19c2b0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:46.989644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:46.990184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:46.990464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:46.990739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:46.992362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:46.999388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:47.082382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:47.084279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76329c7-72b2-4808-9aee-89c0d3bd1478 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:47.103220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 712/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:47.912117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:47.912703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:47.912703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:47.914198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:47.919125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33379b1-d174-41ea-8eb5-b0e05cc2bc3c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:47.919505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33379b1-d174-41ea-8eb5-b0e05cc2bc3c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:47.919676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33379b1-d174-41ea-8eb5-b0e05cc2bc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:47.994799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f33379b1-d174-41ea-8eb5-b0e05cc2bc3c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:47.996022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f33379b1-d174-41ea-8eb5-b0e05cc2bc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:48.062936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:48.093317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:48.250022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:48.251491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:48.270896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:48.397599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab68eef-a3d2-432c-8133-749b84f06801 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:48.484469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:48.513505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:48.514048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:48.514325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:48.514599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:48.516207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:48.605006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:48.606623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e08c1-1d4c-4767-a67e-7b8e5461bd04 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:48.625125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 721/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:49.422556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:49.423104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:49.423104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:49.424484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:49.429623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93559-f0d2-4ef6-a363-9b87ec657bd9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:49.430050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93559-f0d2-4ef6-a363-9b87ec657bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:49.430180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a93559-f0d2-4ef6-a363-9b87ec657bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:49.506741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d2a93559-f0d2-4ef6-a363-9b87ec657bd9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:49.507933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d2a93559-f0d2-4ef6-a363-9b87ec657bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:49.551764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:49.607098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:49.760045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:49.762177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:49.912751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577cb2e-5b89-450e-adcc-a619046dbe25 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:49.966513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:50.025730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:50.026283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:50.026565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:50.026841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:50.028476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:50.119414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:50.121048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60fd71-05ce-4b64-a483-f030454a02ea service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:50.139120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 715/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:50.953475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:50.954056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:50.954056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:50.955483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:50.960822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95335-0e4e-4876-9b8d-68eac3603942 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:50.961303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95335-0e4e-4876-9b8d-68eac3603942 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:50.961406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf95335-0e4e-4876-9b8d-68eac3603942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:51.034746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bcf95335-0e4e-4876-9b8d-68eac3603942 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:51.036530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcf95335-0e4e-4876-9b8d-68eac3603942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:51.038555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:51.134520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:51.285480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:51.286905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:51.432322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7eccd-ce4d-4cd1-bd14-79c92feeb7b3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:51.459744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:51.544584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:51.545163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:51.545442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:51.545711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:51.547313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:51.634940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:51.636542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4fa3a-fc5d-420c-b855-36134f1f7ee2 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:51.655162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 724/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:53.588777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:53.589367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:53.589367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:53.590728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:53.596027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf54b-9281-41ca-aa1a-2c634ec94c4d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:53.596481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf54b-9281-41ca-aa1a-2c634ec94c4d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:53.596600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bf54b-9281-41ca-aa1a-2c634ec94c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:53.637367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:53.662192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-277bf54b-9281-41ca-aa1a-2c634ec94c4d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:53.663401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-277bf54b-9281-41ca-aa1a-2c634ec94c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:53.759590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:53.797528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:53.799074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:53.907244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47553cf-dffe-4428-ad9b-54a7b9ba25a9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:53.939185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:53.981477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:53.982089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:53.982369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:53.982644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:53.984258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:54.092897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:54.094367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a5ccd-ebaa-4a41-a50f-0ceee4a96ba5 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:54.112546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 718/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:56.249146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:56.249698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:56.249698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:56.251064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:56.256502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccb26-0a3b-4c87-adcb-94bf197ce4da None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:56.256897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccb26-0a3b-4c87-adcb-94bf197ce4da None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:56.257098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccb26-0a3b-4c87-adcb-94bf197ce4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:56.324231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5e6ccb26-0a3b-4c87-adcb-94bf197ce4da None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:56.325527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5e6ccb26-0a3b-4c87-adcb-94bf197ce4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:56.338205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:56.421430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:56.460171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:56.461747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:56.563149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81a040-3fe2-4ed4-8b2d-b6dc3479e7a6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:56.576137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:56.638019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:56.638633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:56.638912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:56.639188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:56.640803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:56.748606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:56.750109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5a2a0-e958-4edd-9c62-fabf619913cf service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:56.768712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 727/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:01:59.076427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:59.076825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:59.076984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:59.078349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:01:59.084046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc734f-169b-4711-bb4b-734fc1d4fc0a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:59.084502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc734f-169b-4711-bb4b-734fc1d4fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:59.084606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc734f-169b-4711-bb4b-734fc1d4fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:59.131996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:59.149984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-97bc734f-169b-4711-bb4b-734fc1d4fc0a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:01:59.151192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-97bc734f-169b-4711-bb4b-734fc1d4fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:59.249991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:01:59.289990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:01:59.291536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:59.394119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54c28e-d41c-486f-88a2-4111b0f35dee None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:01:59.435983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:01:59.469419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:01:59.470036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:01:59.470318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:01:59.470595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:01:59.472213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:01:59.580909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:01:59.582413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1099-e3ad-40f1-a8ed-25c7769e8a25 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:01:59.600784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 721/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:01.734683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:01.735271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:01.735271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:01.736647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:01.741979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dedf5-8b83-41ac-9bf3-b174b499cc82 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:01.742383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dedf5-8b83-41ac-9bf3-b174b499cc82 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:01.742541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dedf5-8b83-41ac-9bf3-b174b499cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:01.808759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fc6dedf5-8b83-41ac-9bf3-b174b499cc82 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:01.810015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fc6dedf5-8b83-41ac-9bf3-b174b499cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:01.831209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:01.908911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:01.949164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:01.950713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:02.055837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c87228-8940-4ea7-8363-f429d6bad9a0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:02.075800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:02.129697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:02.130312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:02.130597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:02.131138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:02.132773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:02.241909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:02.243375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766731c-807f-463d-ab95-1890c5a4ef78 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:02.261517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 730/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:04.312515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:04.313103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:04.313103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:04.314428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:04.320097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939577-4421-4774-836c-324ca2105542 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:04.320497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939577-4421-4774-836c-324ca2105542 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:04.320731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba939577-4421-4774-836c-324ca2105542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:04.383795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:04.386708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ba939577-4421-4774-836c-324ca2105542 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:04.388558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba939577-4421-4774-836c-324ca2105542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:04.484179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:04.526947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:04.528508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:04.632023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149e3e4-a61f-4c29-ae47-1d2d9202a903 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:04.683014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:04.709633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:04.710280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:04.710563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:04.710840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:04.712467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:04.822707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:04.824147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c4ff5-0ee2-4bb1-9441-4bd7b6dbe8de service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:04.844046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 724/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:07.008594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:07.009174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:07.009312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:07.010620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:07.016060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7346cfa-78b5-411c-a34b-94c6b0208243 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:07.016471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7346cfa-78b5-411c-a34b-94c6b0208243 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:07.016650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7346cfa-78b5-411c-a34b-94c6b0208243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:07.084726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c7346cfa-78b5-411c-a34b-94c6b0208243 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:07.085988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c7346cfa-78b5-411c-a34b-94c6b0208243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:07.114375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:07.182105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:07.220958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:07.222519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:07.325069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6848b-0d0d-41e2-a2f2-5be8ec0ec517 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:07.353350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:07.397925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:07.398536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:07.398814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:07.399111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:07.400716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:07.510262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:07.511701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef74cb3-1b7b-4b55-ab63-70f54ab21a40 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:07.529292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 733/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:09.615671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:09.616259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:09.616259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:09.617609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:09.622883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e68f-0af6-4e6b-8a62-6ebc6dfcdac2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:09.623286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e68f-0af6-4e6b-8a62-6ebc6dfcdac2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:09.623442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e68f-0af6-4e6b-8a62-6ebc6dfcdac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:09.689636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-20c6e68f-0af6-4e6b-8a62-6ebc6dfcdac2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:09.691034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-20c6e68f-0af6-4e6b-8a62-6ebc6dfcdac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:09.700182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:09.787945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:09.825270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:09.826795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:09.929264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aacad-2e11-4d62-9432-dee79a0f0030 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:09.984270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:10.002453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:10.003070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:10.003351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:10.004174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:10.005853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:10.126388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:10.127817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b640-a451-4cd6-a2d9-ed82c87fd35a service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:10.147025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 727/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:12.204838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:12.205535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:12.205535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:12.206808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:12.212169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301f34-f6dd-4dc4-903b-dcd0906f3701 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:12.212592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301f34-f6dd-4dc4-903b-dcd0906f3701 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:12.212738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7301f34-f6dd-4dc4-903b-dcd0906f3701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:12.279638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b7301f34-f6dd-4dc4-903b-dcd0906f3701 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:12.280868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b7301f34-f6dd-4dc4-903b-dcd0906f3701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:12.320220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:12.380152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:12.417824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:12.419362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:12.522424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136787c4-ac22-418e-b773-7989b5d0c196 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:12.561772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:12.598489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:12.599497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:12.599791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:12.600074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:12.602205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:12.712593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:12.714060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb9929-1899-48ad-817c-749c08c8a172 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:12.731666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 736/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:14.877750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:14.878293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:14.878293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:14.879770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:14.885206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b508f-d1a2-45d2-bfa7-ab3d317b239b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:14.885644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b508f-d1a2-45d2-bfa7-ab3d317b239b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:14.885743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b508f-d1a2-45d2-bfa7-ab3d317b239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:14.951162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cf1b508f-d1a2-45d2-bfa7-ab3d317b239b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:14.952378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cf1b508f-d1a2-45d2-bfa7-ab3d317b239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:14.970887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:15.048499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:15.085309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:15.086853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:15.189623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a1822-9a3c-4a51-af1d-bbf0562568de None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:15.256254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:15.263377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:15.263995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:15.264279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:15.264557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:15.266233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:15.375528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:15.376974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c6878-9d26-40f1-9f63-2f20fba09cd1 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:15.394684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 730/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:17.465068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:17.465638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:17.465638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:17.467070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:17.473112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359d0b-7de1-4056-8680-2d987fbefcbb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:17.473633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359d0b-7de1-4056-8680-2d987fbefcbb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:17.473633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359d0b-7de1-4056-8680-2d987fbefcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:17.537711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6c359d0b-7de1-4056-8680-2d987fbefcbb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:17.538916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6c359d0b-7de1-4056-8680-2d987fbefcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:17.589434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:17.638241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:17.675929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:17.677491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:17.779680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccadf6-71ba-40a7-9973-76933466c748 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:17.832685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:17.859173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:17.859790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:17.860072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:17.860349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:17.862014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:17.967509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:17.968936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eedc8-ac60-459a-b6f6-d0ad4174fbc7 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:17.986900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 739/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:27.636191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=22 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:27.984763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=66 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:37.760077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0204339-b33c-404b-bd28-e5007d9f21bd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:37.760638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0204339-b33c-404b-bd28-e5007d9f21bd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:37.760638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0204339-b33c-404b-bd28-e5007d9f21bd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:37.762014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 732/1471] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:37.767038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06689ff-b890-4d1a-a892-576ae3c3b19a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:37.767443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06689ff-b890-4d1a-a892-576ae3c3b19a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:37.767633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06689ff-b890-4d1a-a892-576ae3c3b19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:37.836206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c06689ff-b890-4d1a-a892-576ae3c3b19a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:37.837364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c06689ff-b890-4d1a-a892-576ae3c3b19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:37.931992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 740/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:38.014940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:38.015219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:38.015490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:38.016817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 733/1473] 10.222.0.65 () {54 vars in 966 bytes} [Wed Jul 27 20:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:38.021688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f965b7-ceb5-4df5-99f6-26d4878167c1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:38.021980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f965b7-ceb5-4df5-99f6-26d4878167c1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:38.022261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f965b7-ceb5-4df5-99f6-26d4878167c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:38.099264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b4f965b7-ceb5-4df5-99f6-26d4878167c1 None None] MFA Rules not processed for user `d4256769bf5b4e81a15b985039cdbd80`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:38.100386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b4f965b7-ceb5-4df5-99f6-26d4878167c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:38.120277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:38.174980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 741/1474] 10.222.0.65 () {58 vars in 1063 bytes} [Wed Jul 27 20:02:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:38.182555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:38.184059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:38.287353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f036c7-1b12-4936-a9e5-a4b2d1541ed6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:38.358852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:38.359386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:38.359663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:38.359933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:38.361563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:38.415675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:38.470021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:38.471528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4ca92-a47b-4345-b799-be2b03652c11 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:38.490935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 734/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:40.606969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:40.607424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:40.607528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:40.608904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 742/1476] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:40.614902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc497f-e472-4bc4-af3b-fa0fcd0594f5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:40.615303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc497f-e472-4bc4-af3b-fa0fcd0594f5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:40.615460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc497f-e472-4bc4-af3b-fa0fcd0594f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:40.681592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-01fc497f-e472-4bc4-af3b-fa0fcd0594f5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:40.682810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-01fc497f-e472-4bc4-af3b-fa0fcd0594f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:40.778419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 735/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:40.804873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:40.806449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:40.940213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917681c-fe02-4fa7-a27c-ff155387ff5c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:40.992352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:41.041016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:41.041642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:41.041924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:41.042201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:41.042907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:41.044667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:02:41.044667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:02:41.044667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:02:41.045855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ef47c-6961-4d68-96f6-a7514c7ae5b0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:41.184621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 743/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Jul 27 20:02:40 2022] GET /identity/v3/projects => generated 2553 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:02:43.193049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:43.193604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:43.193604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:43.194991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 736/1479] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:43.200204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44cb0-c939-4ca1-a2ff-1872d9d30c8c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:43.200647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44cb0-c939-4ca1-a2ff-1872d9d30c8c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:43.200752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44cb0-c939-4ca1-a2ff-1872d9d30c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:43.266362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-afb44cb0-c939-4ca1-a2ff-1872d9d30c8c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:43.267598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-afb44cb0-c939-4ca1-a2ff-1872d9d30c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:43.363203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 744/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:43.406004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:43.407564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:43.417845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:43.552003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315afed-0cba-4a30-b661-6467cb30b129 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:43.630838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:43.664352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:43.664990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:43.665273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:43.665546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:43.667170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:43.756616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:43.758236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebad640-d91f-4c53-af03-caf469439ec5 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:43.776401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 737/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:45.981649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:45.982189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:45.982189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:45.983588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 745/1482] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 20:02:45.988881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580f2a-0ab3-4e57-99c1-f44105d71b3f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:45.989322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580f2a-0ab3-4e57-99c1-f44105d71b3f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:45.989481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580f2a-0ab3-4e57-99c1-f44105d71b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:46.054680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-01580f2a-0ab3-4e57-99c1-f44105d71b3f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:46.055911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-01580f2a-0ab3-4e57-99c1-f44105d71b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:46.102993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:46.153923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 738/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:46.189759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:46.191832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:46.215375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:46.294464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b145-7321-419c-a4bc-08c6484e9c03 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:46.366590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:46.367202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:46.367482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:46.367756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:46.369404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:46.442614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:46.477760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:46.479201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52e32f-0eb2-44a7-a6dc-aa3e076cfde8 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:46.498055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 746/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:48.557627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:48.558175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:48.558175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:48.559542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 739/1485] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:48.564789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc770a-c4dc-4f52-8a59-7b1d8c02d528 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:48.565325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc770a-c4dc-4f52-8a59-7b1d8c02d528 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:48.565435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc770a-c4dc-4f52-8a59-7b1d8c02d528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:48.631599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b2cc770a-c4dc-4f52-8a59-7b1d8c02d528 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:48.632798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b2cc770a-c4dc-4f52-8a59-7b1d8c02d528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:48.726011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 747/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:48.764223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:48.765784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:48.875783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d850114-a395-41b8-bed5-3343be69e51b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:48.947002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:48.948889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:48.949578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:48.950054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:48.950526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:48.952439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:49.059910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:49.061307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b68dd-e58f-4a75-af07-aca0216f2d15 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:49.080075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 740/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:51.227779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50cdd-9402-4827-b325-233787499345 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:51.228430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50cdd-9402-4827-b325-233787499345 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:51.228430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50cdd-9402-4827-b325-233787499345 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:51.229782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 748/1488] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:51.235100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ce6-bb7f-445c-ad0d-c568b1d4abe2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:51.235533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ce6-bb7f-445c-ad0d-c568b1d4abe2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:51.235649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ce6-bb7f-445c-ad0d-c568b1d4abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:51.301960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fbc38ce6-bb7f-445c-ad0d-c568b1d4abe2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:51.303211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fbc38ce6-bb7f-445c-ad0d-c568b1d4abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:51.357264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:51.402396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 741/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:02:51.443762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50cdd-9402-4827-b325-233787499345 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:51.445359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aae50cdd-9402-4827-b325-233787499345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:51.481214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:51.548031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50cdd-9402-4827-b325-233787499345 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:51.620871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:51.621437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:51.621716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:51.621989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:51.623669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:51.702182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:51.731047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:51.732487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b27f6d-d784-4563-a190-a817cdd6186c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:51.750729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 749/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:02:53.949324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:53.949887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:53.949887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:53.951235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 742/1491] 10.222.0.65 () {54 vars in 926 bytes} [Wed Jul 27 20:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:02:53.956459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b091e-fe07-4926-a69c-09d5d77f88e2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:53.956938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b091e-fe07-4926-a69c-09d5d77f88e2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:53.957346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b091e-fe07-4926-a69c-09d5d77f88e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:54.024005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-363b091e-fe07-4926-a69c-09d5d77f88e2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:02:54.025318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-363b091e-fe07-4926-a69c-09d5d77f88e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:54.120597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 750/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Jul 27 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:02:54.160343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:02:54.161972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:54.267162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6f0a4-a8d6-496a-a7dc-7d3c6607f12b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:02:54.341625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:02:54.342238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:02:54.342516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:02:54.342789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:02:54.344407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:02:54.354642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:02:54.456190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:02:54.458038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402ee4c-3b1e-4ce5-a52b-4200d75ed30e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:02:54.477128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 743/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:03:04.118771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=82 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:03:04.474796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=50 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:21.802321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:21.802881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:21.802881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:21.877445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:21.878646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:21.972725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 751/1494] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:21.981109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:21.981627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:21.981747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.058972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:22.060093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.154880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 744/1495] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:22.160060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.161290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.172494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42220038-fe06-463d-9067-f8d07173fbe7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:22.236514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:22.269903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:22.270517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:22.270792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:22.271061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.271744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:04:22.272045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:22.273077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:22.273077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:04:22.273077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:04:22.273960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:22.285512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 752/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:04:22.293973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.307874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd899-6e77-4433-9022-5a5cad70f43d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:22.369918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:22.404112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:22.404727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:22.405036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:22.405316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.405982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:22.408322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:22.408322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:04:22.408322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:04:22.409555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:22.465490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 745/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:04:22 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:04:22.469888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.472196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.505921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c69f5d-3afe-4321-86eb-d03cee723c94 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:22.597204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:22.597813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:22.598089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:22.598357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.599000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:22.599998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:22.599998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:04:22.599998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:04:22.601280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:22.641970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 753/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:04:22 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:04:22.646332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.647933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.717970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb98f-ffa2-4aa3-b5cd-1385d1d22c13 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:22.802395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:22.813526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:22.814279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:22.814563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:22.814844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.815535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:22.816578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:22.816578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:04:22.816578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:04:22.817503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:22.829964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 746/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:04:22.833943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.835462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.847404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a00d0-1655-40cc-82ba-cd1c06a92b53 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:22.872215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:22.941768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:22.942305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:22.942581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:22.943100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:22.943787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:22.944785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:22.944785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:04:22.944785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:04:22.945695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b7396-1faa-4494-9b15-a00f4ebe282d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:22.957727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 754/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:04:22.962040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:22.963576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:22.998601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2878b-2500-4603-af1e-728682f8e033 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:23.100197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:23.100872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.101185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.101467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] PATH_INFO: `/v3/projects/15a9c91f5d3b45e686c268943c97c70e/users/d39374f328304169a53f4d75f6201f9a/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.123380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a9c91f5d3b45e686c268943c97c70e, user_id=d39374f328304169a53f4d75f6201f9a, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:23.124418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:23.124418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:04:23.124418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:04:23.126538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0321ef0-d9b5-42a7-bc60-6ceda8b10c66 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:23.155116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 747/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:04:22 2022] PUT /identity/v3/projects/15a9c91f5d3b45e686c268943c97c70e/users/d39374f328304169a53f4d75f6201f9a/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:04:23.168002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c75cf-1ad6-4f8c-a7b1-2eff39bd19c5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.168303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c75cf-1ad6-4f8c-a7b1-2eff39bd19c5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.168566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c75cf-1ad6-4f8c-a7b1-2eff39bd19c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.224818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-576c75cf-1ad6-4f8c-a7b1-2eff39bd19c5 None None] MFA Rules not processed for user `d39374f328304169a53f4d75f6201f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:23.226002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-576c75cf-1ad6-4f8c-a7b1-2eff39bd19c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:23.303505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 755/1502] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:23.316675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.316993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.317278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.318928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 748/1503] 10.222.0.65 () {54 vars in 964 bytes} [Wed Jul 27 20:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:23.323820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2727d-dfaa-4829-8578-90168d445dda None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.324117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2727d-dfaa-4829-8578-90168d445dda None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.324402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2727d-dfaa-4829-8578-90168d445dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.356247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:23.400323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8bc2727d-dfaa-4829-8578-90168d445dda None None] MFA Rules not processed for user `e5c57fe9a7f141458ac3d01bfa045695`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:23.401560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bc2727d-dfaa-4829-8578-90168d445dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:23.482016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 756/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Jul 27 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:23.489993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:23.491559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:23.542587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:23.597811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373ecd4-0edd-449c-934e-f0f5e6757f3d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:23.668805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:23.669448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.669730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.670006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.671623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:23.769328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:23.770689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:23.774147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:23.788470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 749/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:04:23.834652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b076a53-fcf0-4087-9ef0-c4a45e2f9d8e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:23.835090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b076a53-fcf0-4087-9ef0-c4a45e2f9d8e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:23.835244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b076a53-fcf0-4087-9ef0-c4a45e2f9d8e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:23.836521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 757/1506] 10.222.0.65 () {52 vars in 830 bytes} [Wed Jul 27 20:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:04:23.840065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:23.841344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:23.938627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215bedc-b8cd-4f1b-b29e-243ddce20d4c service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:24.032938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:24.033620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:24.033761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:24.034035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] PATH_INFO: `/v3/users/d39374f328304169a53f4d75f6201f9a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:24.046221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39374f328304169a53f4d75f6201f9a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:24.047269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:24.047269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:04:24.047269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:04:24.049247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:24.058855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:24.115928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39374f328304169a53f4d75f6201f9a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:24.125130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39374f328304169a53f4d75f6201f9a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:24.143083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39374f328304169a53f4d75f6201f9a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:24.154748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39374f328304169a53f4d75f6201f9a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:24.155189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39374f328304169a53f4d75f6201f9a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:24.161314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 750/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:04:23 2022] DELETE /identity/v3/users/d39374f328304169a53f4d75f6201f9a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:04:24.165759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:24.166046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:24.166325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:24.223369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:04:24.224573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:24.300626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:24.313226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 758/1508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:04:24.323425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:24.324659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:24.427799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523f039-59c9-47b4-a7b7-8d03685f648b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:24.481136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:24.502911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:24.503525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:24.503806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:24.504082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:24.505742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:24.612925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:24.614303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e18ce-6d90-4b1b-81dc-55b1a29b489a service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:24.631939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 751/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:04:25.005091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:04:25.006338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:04:25.018624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520a0da-6e5c-4bbc-bf9a-9d1448cec19b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:04:25.120221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:04:25.120777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:04:25.121100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:04:25.121380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] PATH_INFO: `/v3/projects/15a9c91f5d3b45e686c268943c97c70e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:04:25.131222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a9c91f5d3b45e686c268943c97c70e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:04:25.132249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:04:25.132249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:04:25.132249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:04:25.133493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:04:25.189711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a9c91f5d3b45e686c268943c97c70e', 'request_id': 'req-d99b7acd-b739-407b-94e3-f7816fbec6b8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:25.195871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a9c91f5d3b45e686c268943c97c70e', 'request_id': 'req-d99b7acd-b739-407b-94e3-f7816fbec6b8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:25.208978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a9c91f5d3b45e686c268943c97c70e', 'request_id': 'req-d99b7acd-b739-407b-94e3-f7816fbec6b8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:25.234623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] The token cache is being invalidate because project 15a9c91f5d3b45e686c268943c97c70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:04:25.234918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:25.235348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d99b7acd-b739-407b-94e3-f7816fbec6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:04:25.237424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 759/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:04:25 2022] DELETE /identity/v3/projects/15a9c91f5d3b45e686c268943c97c70e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:04:34.629643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=62 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:04:35.232657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:06.748061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:06.748629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:06.748629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:06.749948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:06.754849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d967-1c6b-4819-ac5e-fc6b4235630a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:06.755822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d967-1c6b-4819-ac5e-fc6b4235630a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:06.755974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d967-1c6b-4819-ac5e-fc6b4235630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:06.836014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a5a5d967-1c6b-4819-ac5e-fc6b4235630a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:06.837231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a5a5d967-1c6b-4819-ac5e-fc6b4235630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:06.940910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:07.096690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:07.098896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:07.216755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9649ff-0af4-4433-82be-4edfb0338ea6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:07.222313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:07.223004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:07.223268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:07.223546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:07.231669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:07.233304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a05c-76c6-4d20-b3e6-b2be31e51a95 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:07.243082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 753/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:08.069885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc427a-e55e-45c1-b22a-dd571af66367 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.070461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc427a-e55e-45c1-b22a-dd571af66367 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.070461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc427a-e55e-45c1-b22a-dd571af66367 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.071838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:08.076885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97849-165c-4913-9f56-a53237bc44d5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.077341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97849-165c-4913-9f56-a53237bc44d5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.077480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b97849-165c-4913-9f56-a53237bc44d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.097658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-03b97849-165c-4913-9f56-a53237bc44d5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:08.098850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-03b97849-165c-4913-9f56-a53237bc44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:08.107319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:08.258684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc427a-e55e-45c1-b22a-dd571af66367 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:08.273589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc427a-e55e-45c1-b22a-dd571af66367 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:08.279159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:08.279754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.280029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.280301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.288438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:08.290051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138e11d-31d3-4f79-a169-2c2c09519e4e service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:08.299358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 762/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:08.766296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddf95-533a-493f-b665-60b5508c7825 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.766587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddf95-533a-493f-b665-60b5508c7825 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.766872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ddf95-533a-493f-b665-60b5508c7825 None None] PATH_INFO: `/v3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.768162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 755/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Jul 27 20:06:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:06:08.774351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976ee9-b767-47fa-acff-61ef9af6a165 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:08.774636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976ee9-b767-47fa-acff-61ef9af6a165 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:08.774911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976ee9-b767-47fa-acff-61ef9af6a165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:08.836240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-10976ee9-b767-47fa-acff-61ef9af6a165 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:08.837504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10976ee9-b767-47fa-acff-61ef9af6a165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:08.854723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:08.909751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 763/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 20:06:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:06:17.966558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda082a0-103b-4622-9642-7e3b0e7645a0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:17.967082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda082a0-103b-4622-9642-7e3b0e7645a0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:17.967082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda082a0-103b-4622-9642-7e3b0e7645a0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:17.968357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 756/1519] 10.222.0.65 () {54 vars in 961 bytes} [Wed Jul 27 20:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:17.973408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5651083-f238-4523-b720-c955daf50b22 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:17.973863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5651083-f238-4523-b720-c955daf50b22 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:17.973970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5651083-f238-4523-b720-c955daf50b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:17.995455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e5651083-f238-4523-b720-c955daf50b22 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:17.996658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e5651083-f238-4523-b720-c955daf50b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:18.005219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 764/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Jul 27 20:06:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:18.013414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda082a0-103b-4622-9642-7e3b0e7645a0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:18.026946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda082a0-103b-4622-9642-7e3b0e7645a0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:18.031960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:18.032567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:18.032851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:18.033192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:18.043400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:18.045650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc8cd3-95a5-4653-a631-a4be628ddb56 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:18.057468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 757/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:19.538803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a64f3-d113-4d12-8805-94333fef6702 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:19.539372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a64f3-d113-4d12-8805-94333fef6702 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:19.539372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a64f3-d113-4d12-8805-94333fef6702 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:19.540811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:19.545742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea9408-8e15-4f89-9062-db9769292587 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:19.546173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea9408-8e15-4f89-9062-db9769292587 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:19.546275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea9408-8e15-4f89-9062-db9769292587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:19.565858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8bea9408-8e15-4f89-9062-db9769292587 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:19.567148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8bea9408-8e15-4f89-9062-db9769292587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:19.575317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:06:19.728064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a64f3-d113-4d12-8805-94333fef6702 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:19.744703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a64f3-d113-4d12-8805-94333fef6702 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:19.750258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:19.750855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:19.751135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:19.751412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:19.760401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:19.762023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a82466-6867-4976-9fb3-7c191d01fc71 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:19.771464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 766/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:20.602312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309007e2-7ea4-42bb-834e-5c6977ed988d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:20.602885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309007e2-7ea4-42bb-834e-5c6977ed988d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:20.602885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309007e2-7ea4-42bb-834e-5c6977ed988d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:20.604207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 20:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:20.609026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e09c-4dd3-46c0-8959-c96bc42d3bdc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:20.609423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e09c-4dd3-46c0-8959-c96bc42d3bdc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:20.609573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3e09c-4dd3-46c0-8959-c96bc42d3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:20.628885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c4d3e09c-4dd3-46c0-8959-c96bc42d3bdc None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:20.630101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c4d3e09c-4dd3-46c0-8959-c96bc42d3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:20.638593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 20:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:20.793292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309007e2-7ea4-42bb-834e-5c6977ed988d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:20.808138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309007e2-7ea4-42bb-834e-5c6977ed988d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:20.813612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:20.814211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:20.814495 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:20.814772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:20.823762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:20.825384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876836b-051a-472a-82ce-72310aec2a4e service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:20.834682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 760/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:24.622243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:24.622815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:24.622815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:24.624146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:24.629286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e00ee-ebcf-4818-a0b6-ce0c6705f570 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:24.629735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e00ee-ebcf-4818-a0b6-ce0c6705f570 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:24.629871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e00ee-ebcf-4818-a0b6-ce0c6705f570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:24.649583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-810e00ee-ebcf-4818-a0b6-ce0c6705f570 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:24.650786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-810e00ee-ebcf-4818-a0b6-ce0c6705f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:24.658616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:24.736061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:24.737811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:24.814916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce866ec-30ae-428a-a4e8-06e1fab09849 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:24.818986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:24.819597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:24.819880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:24.820158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:24.829087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:24.830521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d0fd-ecbd-43ed-bf99-ab73ae04596d service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:24.840148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 769/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:26.863250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:26.863837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:26.863837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:26.865702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:26.871361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0299a-c76f-4e7a-8ae5-1fc687fbf233 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:26.871863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0299a-c76f-4e7a-8ae5-1fc687fbf233 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:26.871970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0299a-c76f-4e7a-8ae5-1fc687fbf233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:26.892435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dea0299a-c76f-4e7a-8ae5-1fc687fbf233 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:26.893647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dea0299a-c76f-4e7a-8ae5-1fc687fbf233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:26.901391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:26.938146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:26.939879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:26.953420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57a7ed-5de4-4d10-83d4-1b964dce594d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:26.957002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:26.957541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:26.957828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:26.958105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:26.966942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:26.968310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9040e61-3701-40f6-95a3-4680e7430e3b service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:26.977517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 763/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:30.535629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:30.536208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:30.536208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:30.538185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:30.543505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea12f5-1134-4942-94ef-547a66bf7b31 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:30.543978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea12f5-1134-4942-94ef-547a66bf7b31 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:30.544083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea12f5-1134-4942-94ef-547a66bf7b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:30.562982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a6ea12f5-1134-4942-94ef-547a66bf7b31 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:30.564169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a6ea12f5-1134-4942-94ef-547a66bf7b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:30.572140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:06:30.885409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:30.887156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:30.952332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:30.966309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef56d34-5811-4725-8326-7e0e349046ab None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:30.969879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:30.970499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:30.970784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:30.971060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:30.980025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:30.981505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2900de-4fdd-402d-b112-ca3b0c6804b1 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:30.990938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 772/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:32.998944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:32.999466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:32.999466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:33.000735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 20:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:06:33.006458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31308f1-3109-4b2b-a663-ef5174dba622 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:33.006875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31308f1-3109-4b2b-a663-ef5174dba622 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:33.007009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31308f1-3109-4b2b-a663-ef5174dba622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:33.026671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e31308f1-3109-4b2b-a663-ef5174dba622 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:06:33.027870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e31308f1-3109-4b2b-a663-ef5174dba622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:33.035699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 20:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:06:33.346138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:06:33.347822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:06:33.360666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac7330-8248-4486-92aa-1c675e177357 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:06:33.364141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:06:33.365378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:06:33.365624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:06:33.365896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:06:33.374572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:06:33.375994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289fd71-1c09-4286-818e-c10ad4dbaec7 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:06:33.387838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 766/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:06:43.021765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:06:43.381883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:03.434202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.435433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.435933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.469566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.471321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.485333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 774/1540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:03.495056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.495639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.495639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.534344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.535738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.548345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 767/1541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:03.553511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:03.571632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9fd76-06b9-4a69-b5ec-fc3ce9c2abcb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:03.581696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:03.582754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.583260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.584373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.585355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:03.585902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:03.587947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2022288-464b-47b8-b6df-dacdec459f83 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:03.608716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 775/1542] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:03.617782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:03.630166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed657f-d263-449c-8322-e3a2e870946d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:03.639515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:03.640163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.640450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.640729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.641452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:03.643402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:03.682435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc325-07dc-49fd-932b-220e3be40a78 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.683109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc325-07dc-49fd-932b-220e3be40a78 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.683588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdc325-07dc-49fd-932b-220e3be40a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.707669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 768/1543] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/projects => generated 389 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:03.714080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:03.723190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-14bdc325-07dc-49fd-932b-220e3be40a78 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.724934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-14bdc325-07dc-49fd-932b-220e3be40a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.806340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de4385-ff2c-4cb0-be91-948b084bb055 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:03.821739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:03.822401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.822691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.822962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.824169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:03.825239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:03.825239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:08:03.825239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:08:03.826477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bff19-b357-4620-a38c-4eb21764261a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:03.849190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 776/1544] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:03.851109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8718ba-ae3a-4231-b970-1420a387cd24 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.851625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8718ba-ae3a-4231-b970-1420a387cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.852225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8718ba-ae3a-4231-b970-1420a387cd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.861286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 769/1545] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:03.862608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.862911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.863190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.906039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.907789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.918258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0c8718ba-ae3a-4231-b970-1420a387cd24 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.919706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c8718ba-ae3a-4231-b970-1420a387cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.922005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 770/1546] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:03.922868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:03.936588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 777/1547] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:03.937339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da39fcf-9176-47eb-be30-24f5351e7f8c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:03.937973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.938262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.938537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.952344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:03.953386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.954161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.954650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.955591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:03.957848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e3642-980f-4151-923d-655adf1678b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:03.963918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.965163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.969299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 771/1548] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:03.972548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3292-9bae-4a8f-820b-be90c60ac648 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.972548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3292-9bae-4a8f-820b-be90c60ac648 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.972548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3292-9bae-4a8f-820b-be90c60ac648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.975676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 778/1549] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:03.979334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:03.990908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10280b98-7564-4f59-997d-2f9bdd9161ff None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:03.991830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7bef3292-9bae-4a8f-820b-be90c60ac648 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:03.993085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7bef3292-9bae-4a8f-820b-be90c60ac648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:03.996023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:03.996917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:03.997401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:03.997838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:03.998797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:03.999363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.001140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 772/1550] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.001449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0ba2f-af97-4954-853f-fe95ef469e08 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.002484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf087b-769e-4713-8a80-8bd6e12632b1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.002776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf087b-769e-4713-8a80-8bd6e12632b1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.003056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf087b-769e-4713-8a80-8bd6e12632b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.018330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 779/1551] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.020351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.020934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.021940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.032771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8cbf087b-769e-4713-8a80-8bd6e12632b1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.034032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8cbf087b-769e-4713-8a80-8bd6e12632b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.042830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 773/1552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.044151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.044284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740556f7-7a8f-4790-acf9-901f3def1784 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.044468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740556f7-7a8f-4790-acf9-901f3def1784 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.044723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740556f7-7a8f-4790-acf9-901f3def1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.045530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.054821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 780/1553] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.055684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.064943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-740556f7-7a8f-4790-acf9-901f3def1784 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.065141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2ffe1-f34f-48cc-b4b7-f6eee6cffe13 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.066175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-740556f7-7a8f-4790-acf9-901f3def1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.073637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.074427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.078789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.079530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 774/1554] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.079657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.079657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.079834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.079834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.079834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.080080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57b85a-da63-46b7-961b-db2236d681fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.090439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 781/1555] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:04.092097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac3f33-67d0-4ba0-9e68-7f1149fa3b19 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.092557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac3f33-67d0-4ba0-9e68-7f1149fa3b19 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.093021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac3f33-67d0-4ba0-9e68-7f1149fa3b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.099797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.101053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.109330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 775/1556] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.110230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.116747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1fac3f33-67d0-4ba0-9e68-7f1149fa3b19 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.118015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1fac3f33-67d0-4ba0-9e68-7f1149fa3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.121788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e0890-91d1-42ef-926a-39b235797947 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.127534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.127916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 782/1557] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.128147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.128436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.128783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.129172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5257f5-ac7f-490c-be4a-0095e54c5d86 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.129460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5257f5-ac7f-490c-be4a-0095e54c5d86 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.129577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.129748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5257f5-ac7f-490c-be4a-0095e54c5d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.131685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.149727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-db5257f5-ac7f-490c-be4a-0095e54c5d86 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.151301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-db5257f5-ac7f-490c-be4a-0095e54c5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.160510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 783/1558] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.161141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 776/1559] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.162461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364111b5-c0c6-4d86-bdbb-c04a562f9979 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.162969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364111b5-c0c6-4d86-bdbb-c04a562f9979 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.163288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364111b5-c0c6-4d86-bdbb-c04a562f9979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.166081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.182567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-364111b5-c0c6-4d86-bdbb-c04a562f9979 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.183825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-364111b5-c0c6-4d86-bdbb-c04a562f9979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.216158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:04.231454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0267a7-c4cb-459b-ab36-bf21afbce5eb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.236282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.237035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.237325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.237639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.238335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.238645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.239735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:04.239735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:08:04.239735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:08:04.241009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.246071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 784/1560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.247405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.248242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.248242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.253039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 777/1561] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.253828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.267174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.268372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.268510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35020457-418e-4be4-bc4c-095e5f97cd31 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.273511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.274188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.274465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.274739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] PATH_INFO: `/v3/projects/ec01b599458b451e8d04a428c1c33513/users/b734a345bfe5483d9a5a1e35d89c935b/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.277400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 785/1562] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.278188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.288553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc591355-70c0-4618-85b3-169a054edfb5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.293293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.293811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec01b599458b451e8d04a428c1c33513, user_id=b734a345bfe5483d9a5a1e35d89c935b, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.293943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.294161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.294439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.295130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.295437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.296560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.297045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.315986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 786/1563] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.315986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.317206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:04.317464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 778/1564] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:04 2022] PUT /identity/v3/projects/ec01b599458b451e8d04a428c1c33513/users/b734a345bfe5483d9a5a1e35d89c935b/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:04.319042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.324750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e52cee-c646-4ec8-ba49-b5ab1c7ef36b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.381062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.381338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cc496-bd1a-4f2c-ada9-6da014719547 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.381686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.381964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.382234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.382941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.383223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.384819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb0bbf-0c72-44ac-ae9f-5644556718a1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.386183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.386867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.387154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.387554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.388247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.390208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.397298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 787/1565] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.398585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.398769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.399041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.419379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.420387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 779/1566] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.421096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.423930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.430679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06551870-e3d2-4b73-8646-6c3c3b94f046 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.432222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 788/1567] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.433091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.435896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.436236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.436520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.436859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.437533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.437844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.439513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.442590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909825fa-294a-48b3-a8de-96db389e67e4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.448147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.448751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.449072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.449346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.450007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.451022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:04.451022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:08:04.451022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:08:04.452215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.458035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 780/1568] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.458035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.469413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f18a6-255f-41b2-b7d1-72983ac7477d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.474286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.474899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.475180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.475456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.476152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.476459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.478077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c0675-f85a-430e-aea9-10a4e8081da5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.480652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 789/1569] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.484732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.497237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 781/1570] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.497237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.497237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.497237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.511951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:04.513212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:04.547547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819fb65-5f4c-44c7-98ad-99ce4870f83f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.552323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.553174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.553471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.553740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.554398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.556315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.574593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 782/1571] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:04.574593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.585440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3e7d2-1b1b-4393-8cc9-914113dea2a8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.589675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 790/1572] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.589675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.605910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:04.647556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:04.649688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.650562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.651003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.651287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.651972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.652787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3967c-6c2c-4101-a229-78905db583a5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.654021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.658987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.659600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.659883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.660150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.660838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.662520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.672445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 791/1573] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:04.673291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.684009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce9b8-412f-4d1d-a7bb-d76716aac6c2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.684194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 783/1574] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.688141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.741456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.741990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.742271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.742548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.743271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:04.743574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.745201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.750160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a2c18-fa15-4d55-a37d-fd24584dc9d4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.754823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.756126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.756457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.756857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.757564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.758282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 792/1575] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:04.759064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.759488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.768749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46082640-64de-4e71-ae98-3ae22c0f8c68 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.774516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.775128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.775409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.775685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.776347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.778311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.788958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 784/1576] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.789987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.801713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 793/1577] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.803130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.852883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ea3b0-71ba-4238-9a29-5fb6a5bfad23 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.857546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.858171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.858456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.858737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.858858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a736ba-4eb6-4869-b487-01a184e62449 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.859431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.861402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.889433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 785/1578] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:04.890290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.908554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:04.913112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.913673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.914126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.914423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.925209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.927687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.937461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 794/1579] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:04.938870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.949659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefda7c7-6c9d-4c59-96ac-794904d6b952 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.954520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.955142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.955424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.955703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.956403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.958083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.960334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb056b-bd1a-4d0c-8519-de32ce3698d5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.967727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.968673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.969302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.969759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.970480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 795/1580] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:04.970831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.971322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:04.973442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:04.982833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156e34f-a6ee-4797-a448-4ed231cbf9d9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:04.989135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:04.990111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:04.990643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:04.990934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:04.992061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:04.994255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.003377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 786/1581] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.003377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.011865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:05.019349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab03c8f-a296-40de-ba48-a1ee3bc12d8f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.022964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 796/1582] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:04 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.023518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.026921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.027689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.028068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.028522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.029334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.031680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.034124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae531a-a39f-4ec0-a9d5-1ef35bd23945 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.040409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.041190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.041542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.042101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.042645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.044840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.071003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 787/1583] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.071003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.076118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 797/1584] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.076118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.084913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb97c6-8991-49f5-8d0e-dccb120ffc54 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.149330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cd14b-0cda-464a-aa4b-1a1dd97cff18 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.154534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.157304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.157304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.157304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.157304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.158864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.167279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.167936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.168148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.168420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.169035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 788/1585] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.169498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.169831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.171580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.180212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8e3c-b046-4a80-a96f-6f17c84fc23a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.186582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.187418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.187641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.188033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] PATH_INFO: `/v3/projects/8d9a9e3a2b0e4918950cfd454aab9fbb/users/ea3c5c3007e04391b5b108473da62787/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.197650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 798/1586] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.198493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.210755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec08cd-d61a-487f-864c-466642029f53 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.210981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9a9e3a2b0e4918950cfd454aab9fbb, user_id=ea3c5c3007e04391b5b108473da62787, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.214220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.215599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.216212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.216489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.216758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.217486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.219105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.229841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 789/1587] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/8d9a9e3a2b0e4918950cfd454aab9fbb/users/ea3c5c3007e04391b5b108473da62787/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:05.237788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.241233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 799/1588] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.241233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.254583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d7e5e-1b90-449b-b39a-91d04ee136b6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.303300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:05.310351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c59c-92fd-4455-87b7-23f0c34e8ca2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.316714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.318515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.318801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.319079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.319941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.321242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.321870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.322185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.322465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.322597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.323348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.325738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.334461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 790/1589] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.338197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.348238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6d624-d44c-4bab-923d-9448a6c562ac admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.350621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 800/1590] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:05.351495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.353994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.354620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.354906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.355183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.355881 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.357515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.361806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54becd-626a-4a6b-9b8c-d17cc82fc298 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.371374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 791/1591] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.371374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.373453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.374446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.375337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.376570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.377857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.379759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.379887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba71acc-eb58-48ec-ba6f-de6b3ee0446f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.385525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.386149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.386436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.386716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.387418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.389060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb6f5e-9966-4beb-b6e5-a201c4fa0796 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.401680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 792/1592] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.402893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.403189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.403550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.406119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 801/1593] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.407214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.420638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab732a-2438-4c23-8acf-204fe7c40857 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.426885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.428525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.429050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.429533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.430754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.433303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.437541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:05.438967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:05.446207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 802/1594] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.447153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.448423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 793/1595] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:05.449749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.460920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1c902-4cd9-40e7-944b-3400018f0ee6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.466240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dbba1-fd21-41fc-b94d-ca8de2b613cb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.466389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.466909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.467336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.467616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.468464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.470509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.471645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.472356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.472732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.473144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.474498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.476246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.483920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 794/1596] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.485091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.491245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 803/1597] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.491245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.498912 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f780b-23f5-461a-b55e-4177224c946d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.504112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.504782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.505297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.505517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abe75-8f6d-428b-a696-1e244c4071d0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.505852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.506809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.508518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.512784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.513427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.513712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.513994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.514705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.516304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.521679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 795/1598] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.522491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.528605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 804/1599] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.529532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.536916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:05.537811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c439-09f1-4a06-977e-f47e5491dfcd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.543593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770791f-7425-427f-ae62-acaa55f0c3ee admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.544561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.545342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.545625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.545925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] PATH_INFO: `/v3/projects/4fa623a3ef594b2bba664198d0326167/users/64e5a303a6e84995b04087a557437012/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.550380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.551076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.551358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.551636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] PATH_INFO: `/v3/projects/6cab65810db34765966b2ca0332088be/users/ec4599fb9568454ba36b01038c345248/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.566297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa623a3ef594b2bba664198d0326167, user_id=64e5a303a6e84995b04087a557437012, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.567501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cab65810db34765966b2ca0332088be, user_id=ec4599fb9568454ba36b01038c345248, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.568832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:05.568832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:08:05.568832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:08:05.569321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.570835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.584125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 796/1600] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/4fa623a3ef594b2bba664198d0326167/users/64e5a303a6e84995b04087a557437012/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:05.585285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.602171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 805/1601] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/6cab65810db34765966b2ca0332088be/users/ec4599fb9568454ba36b01038c345248/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:05.602171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.653491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087cadb-1b0d-4719-a1a3-98c2fffbda52 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.670306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ec35b-8835-4efa-9af6-5af3f740e976 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.676184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.676926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.678616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.678837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.679675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.682197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.693455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 806/1602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:05.694186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.704516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409031e-62e1-420a-bfad-1e53f4cce1ab admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.707963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.709221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.709506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.709786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.710772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.711552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.711896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.712235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] PATH_INFO: `/v3/projects/730a466023454452a63effcf2672e10c/users/73385c17c60d4a47b504b9fd9892762d/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.719839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.721648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.732932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=730a466023454452a63effcf2672e10c, user_id=73385c17c60d4a47b504b9fd9892762d, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.734250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 797/1603] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:05.735234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.742767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.745506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f622a-adbd-42b6-a229-6fcc6671bc92 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:05.751863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:05.752774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.753048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.753318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] PATH_INFO: `/v3/projects/68afb3e7ea43454fa92d2d21560432f2/users/f851d33b4920425ca3f7e2eec8e72999/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.765367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 807/1604] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/730a466023454452a63effcf2672e10c/users/73385c17c60d4a47b504b9fd9892762d/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:05.766132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:05.800988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68afb3e7ea43454fa92d2d21560432f2, user_id=f851d33b4920425ca3f7e2eec8e72999, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:05.809401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773bded-e10c-4dbd-ad27-6787698ee347 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:05.848389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 798/1605] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/68afb3e7ea43454fa92d2d21560432f2/users/f851d33b4920425ca3f7e2eec8e72999/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:05.851050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60943f34-d4a4-4a64-afba-30df0765b9eb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:05.851718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60943f34-d4a4-4a64-afba-30df0765b9eb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:05.851843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60943f34-d4a4-4a64-afba-30df0765b9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:05.889661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:05.901058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-60943f34-d4a4-4a64-afba-30df0765b9eb None None] MFA Rules not processed for user `64e5a303a6e84995b04087a557437012`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:05.903455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-60943f34-d4a4-4a64-afba-30df0765b9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:05.912590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e2ae-6877-4db6-8fa4-91999538c808 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:06.017783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 799/1606] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.018997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.019285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.019564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.047331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:06.048245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.048531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.048809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] PATH_INFO: `/v3/projects/1626b826d47a47429a947fa36ace64e4/users/a48eaef4d70b4116bd307374d819bca6/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.050585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:06.051820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:06.055332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:06.070691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 800/1607] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.071302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:06.081460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1626b826d47a47429a947fa36ace64e4, user_id=a48eaef4d70b4116bd307374d819bca6, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:06.084977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd72333-9b72-45d6-adf4-8123ceb9d3cf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:06.092731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a6a8-e747-4fb1-9b1c-51056b8c2473 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:06.098731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:06.101595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.102789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.103067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] PATH_INFO: `/v3/projects/8c91238659234e0b8e174b52cd7dba54/users/cf97b651a3e5475c9656c165a24f0d9a/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.109803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 808/1608] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:05 2022] PUT /identity/v3/projects/1626b826d47a47429a947fa36ace64e4/users/a48eaef4d70b4116bd307374d819bca6/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:06.111088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d75cf-8b90-4373-9529-b025ed63ca10 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.111377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d75cf-8b90-4373-9529-b025ed63ca10 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.111656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d75cf-8b90-4373-9529-b025ed63ca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.141400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c91238659234e0b8e174b52cd7dba54, user_id=cf97b651a3e5475c9656c165a24f0d9a, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:06.144125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca66150-9e1c-4f81-9982-06cf45ce1260 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:06.174765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 801/1609] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:06 2022] PUT /identity/v3/projects/8c91238659234e0b8e174b52cd7dba54/users/cf97b651a3e5475c9656c165a24f0d9a/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:06.176451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.176740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.177071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.187686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a61d75cf-8b90-4373-9529-b025ed63ca10 None None] MFA Rules not processed for user `73385c17c60d4a47b504b9fd9892762d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:06.189222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a61d75cf-8b90-4373-9529-b025ed63ca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:06.235797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] MFA Rules not processed for user `f851d33b4920425ca3f7e2eec8e72999`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:06.237194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:06.335129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 802/1610] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.338326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:06.416200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 809/1611] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.419735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.423749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.424060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.462823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:06.467074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:06.547478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac2e8b-2e72-4f9d-9e54-58002d024bc6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:06.562004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:06.562719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.564550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.565406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.569022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:06.571108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a97b9-24ee-4e04-bbdd-49c6e389babb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:06.587364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:06.602945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 810/1612] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.603845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:06.646592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 803/1613] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/projects => generated 389 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:06.648287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50f8c-f412-46a2-9f30-6e57593259a3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.649185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50f8c-f412-46a2-9f30-6e57593259a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.650267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50f8c-f412-46a2-9f30-6e57593259a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.706537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-66f50f8c-f412-46a2-9f30-6e57593259a3 None None] MFA Rules not processed for user `cf97b651a3e5475c9656c165a24f0d9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:06.707910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-66f50f8c-f412-46a2-9f30-6e57593259a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:06.748136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220844a-8a36-4cde-abad-fb908bfe0f14 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:06.811383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:06.870536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:06.871343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.876891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.879008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.903057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 804/1614] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:06.904937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e38e-b079-49e9-b86c-6b038e8b662d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.905249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e38e-b079-49e9-b86c-6b038e8b662d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.905526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183e38e-b079-49e9-b86c-6b038e8b662d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.907924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 805/1615] 10.222.0.65 () {54 vars in 962 bytes} [Wed Jul 27 20:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:06.918307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9823a-f4fa-4d65-b1c0-45d6d3928a47 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.918307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9823a-f4fa-4d65-b1c0-45d6d3928a47 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.918623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9823a-f4fa-4d65-b1c0-45d6d3928a47 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:06.919890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 806/1616] 10.222.0.65 () {54 vars in 962 bytes} [Wed Jul 27 20:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:06.932992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9823a-f4fa-4d65-b1c0-45d6d3928a47 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:06.949930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9823a-f4fa-4d65-b1c0-45d6d3928a47 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:06.954680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:06.955442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:06.955733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:06.956011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.024906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.027306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.039114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 811/1617] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:07.040482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.052576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b639-a2b6-44f2-8edc-d718904e164b service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.059380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.060313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.060813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.061846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.063050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.065277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6e2a-0604-492b-a717-a3e91d6b0fbc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.080759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.082264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.109204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 807/1618] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:07.113882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.119969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 812/1619] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:07 2022] POST /identity/v3/users => generated 301 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:07.124108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.124413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.125019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.139733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aac2cc-f9c7-400a-97e5-0fcaaec18594 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.154536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.155288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.155548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.155828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.156501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.160440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ea228-ceea-4d25-a62d-371ed12e5740 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.218961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:07.220205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:07.225920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 808/1620] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:07 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:07.229008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.229008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.229609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.275418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:07.276761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:07.380624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:07.384754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 813/1621] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Jul 27 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:07.386159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.451636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 809/1622] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Jul 27 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:07.454891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.497902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:07.560283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afa7a6-8dac-4abd-be97-26fa71989eed None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.573422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.574584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.574715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.575000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.575735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.577393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.625843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 814/1623] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:07.627920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.642532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d271633-a58d-4ebf-a483-871abc2aca35 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.653459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.654112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.654395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.654671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.656591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.658597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.660162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6eadda-f693-4b57-94d1-88c11158ebe2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.674494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.675135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.675571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.676156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.711991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 810/1624] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:07 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:07.714165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.752665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d22ed-5697-4ed1-86e4-6abe630feb13 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.769585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.770700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.772719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.774954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.824486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.827554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.857832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 815/1625] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:07.858687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.879620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c559a-3ca4-4571-b749-d04f1bc90aae service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:07.895253 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:07.896840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:07.897290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:07.897589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] PATH_INFO: `/v3/projects/68bccff39d43455bb4846271ce0bebae/users/beab9378ea824814b9f57aa403fac44d/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:07.925108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bccff39d43455bb4846271ce0bebae, user_id=beab9378ea824814b9f57aa403fac44d, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.929328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.953038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 816/1626] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:07 2022] PUT /identity/v3/projects/68bccff39d43455bb4846271ce0bebae/users/beab9378ea824814b9f57aa403fac44d/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:07.954014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:07.960549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:07.967532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:07.988293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 811/1627] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:07.990221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.004463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6917da4-6862-4019-8635-c4e140b94dfc service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.057742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8022b6-a77f-4561-82c0-0b8eb0dc25c5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.064326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.064995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.065299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.065726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.066272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.073890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.091310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 817/1628] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:08.092610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.104195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844f962-489c-4d17-9793-11c9090a5e4c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.111139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.112310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.112597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.112926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] PATH_INFO: `/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1/users/1cd0de6895334360914d96056b608c0f/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.136056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8f8b7b6b7c4f4d8de21b3fe6ce37c1, user_id=1cd0de6895334360914d96056b608c0f, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.138862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.149044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.150112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.150975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.151480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.153525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.162040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 818/1629] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:08 2022] PUT /identity/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1/users/1cd0de6895334360914d96056b608c0f/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:08.162953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.164622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.195460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 812/1630] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:08.200900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.217186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:08.228320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:08.229969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf46082-cd58-46f3-a7aa-8034e1b52a16 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.334540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.335299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4c571-b99e-4727-a12a-01646ad2793f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.335755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.336346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.336799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] PATH_INFO: `/v3/projects/68bccff39d43455bb4846271ce0bebae/users/beab9378ea824814b9f57aa403fac44d/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.340214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bccff39d43455bb4846271ce0bebae, user_id=beab9378ea824814b9f57aa403fac44d, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.344679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff556d9-11dc-42ed-b9e6-aa30f4a408de admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.349490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.350104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.350382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.350654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.351448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.357193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.359596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 813/1631] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:08 2022] PUT /identity/v3/projects/68bccff39d43455bb4846271ce0bebae/users/beab9378ea824814b9f57aa403fac44d/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:08.370367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8741c7-adcd-4413-8f62-2d39f37cda8f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.371149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8741c7-adcd-4413-8f62-2d39f37cda8f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.371681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8741c7-adcd-4413-8f62-2d39f37cda8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.391279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 819/1632] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:08.392192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.415874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865fa37-ef5e-4f57-95a0-8f50cb87189b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:08.517505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ed8741c7-adcd-4413-8f62-2d39f37cda8f None None] MFA Rules not processed for user `beab9378ea824814b9f57aa403fac44d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:08.528244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed8741c7-adcd-4413-8f62-2d39f37cda8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:08.563137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:08.563846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.564129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.564408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] PATH_INFO: `/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1/users/1cd0de6895334360914d96056b608c0f/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.568040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8f8b7b6b7c4f4d8de21b3fe6ce37c1, user_id=1cd0de6895334360914d96056b608c0f, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:08.574217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddabbdd-823c-45ef-ba3b-4a267e626d3f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:08.612627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 820/1633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:08 2022] PUT /identity/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1/users/1cd0de6895334360914d96056b608c0f/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:08.615191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6207e3f-42c0-494f-95b0-681e4507ce5a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.615505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6207e3f-42c0-494f-95b0-681e4507ce5a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.615772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6207e3f-42c0-494f-95b0-681e4507ce5a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.620884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 821/1634] 10.222.0.65 () {54 vars in 912 bytes} [Wed Jul 27 20:08:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:08.627355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.629171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.629563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.685178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 814/1635] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:08.686510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f75aa-c015-4bb9-b7b5-cc5206db4720 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.686861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f75aa-c015-4bb9-b7b5-cc5206db4720 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.690200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f75aa-c015-4bb9-b7b5-cc5206db4720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.704339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] MFA Rules not processed for user `1cd0de6895334360914d96056b608c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:08.705790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:08.722233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4b4f75aa-c015-4bb9-b7b5-cc5206db4720 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:08.728040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4b4f75aa-c015-4bb9-b7b5-cc5206db4720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:08.913312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 822/1636] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:08.913312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:08.963005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 815/1637] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:08.964645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efb22-db3c-46af-a968-27ebcfd60146 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:08.966825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efb22-db3c-46af-a968-27ebcfd60146 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:08.967669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41efb22-db3c-46af-a968-27ebcfd60146 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:08.970921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 816/1638] 10.222.0.65 () {54 vars in 912 bytes} [Wed Jul 27 20:08:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:08.972867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41efb22-db3c-46af-a968-27ebcfd60146 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:09.014752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:09.027643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:09.068472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb2b9-17d8-4a24-93cc-641df78e9999 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:09.087397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:09.091557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:09.091557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:09.091557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:09.118622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41efb22-db3c-46af-a968-27ebcfd60146 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:09.128410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:09.129122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:09.130913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:09.131339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:09.162837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:09.169321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d825ea-b053-472b-8af4-0d5abe7a68e2 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:09.183839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 817/1639] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:09.185277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:09.185607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:09.186584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:09.230376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:09.232048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:09.245992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 823/1640] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:09.277549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:09.280072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:09.329942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 818/1641] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:09.331910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:09.345762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd803c07-983b-45cf-88d4-a669a3c66f8a service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:09.353865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:09.357808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:09.361303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:09.361694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:09.395174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:09.404694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:09.437184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 824/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:09.443962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:09.459167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8778b8-8da0-474e-a785-1269f4c0d083 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:09.469960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:09.471536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:09.472476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:09.473522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:09.515561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:09.520359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:09.554588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 819/1643] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:10.188835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:10.261911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:10.290472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633f18-7e32-406e-8695-96501f6f8586 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:10.305000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:10.305710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:10.305995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:10.306272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] PATH_INFO: `/v3/users/73385c17c60d4a47b504b9fd9892762d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:10.307685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73385c17c60d4a47b504b9fd9892762d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:10.308775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:10.308775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jul 27 20:08:10.308775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: Jul 27 20:08:10.321786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:10.439554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7383a-ef51-4221-8ef7-afef5abb2b13 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:10.441749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73385c17c60d4a47b504b9fd9892762d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.449678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:10.450922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:10.451352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:10.451788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] PATH_INFO: `/v3/users/64e5a303a6e84995b04087a557437012` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:10.453394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e5a303a6e84995b04087a557437012)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:10.458526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:10.494513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73385c17c60d4a47b504b9fd9892762d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.496397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73385c17c60d4a47b504b9fd9892762d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.525245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73385c17c60d4a47b504b9fd9892762d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.534856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:10.549057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10388d02-f1e6-4b2d-b571-3e67f82d023c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73385c17c60d4a47b504b9fd9892762d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.565631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e5a303a6e84995b04087a557437012'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.571615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 825/1644] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:10 2022] DELETE /identity/v3/users/73385c17c60d4a47b504b9fd9892762d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:10.573888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:10.575022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:10.575475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:10.589515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e5a303a6e84995b04087a557437012'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.636714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e5a303a6e84995b04087a557437012'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.644273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:10.653177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:10.665583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e5a303a6e84995b04087a557437012'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.668239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8eac8709-260c-4597-9302-652a865cdd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e5a303a6e84995b04087a557437012'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:10.711966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 820/1645] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:10 2022] DELETE /identity/v3/users/64e5a303a6e84995b04087a557437012 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:10.711966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:10.711966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:10.711966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:10.812549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:10.819015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:10.882238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:10.911259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 826/1646] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:10.911847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:10.913519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:10.941371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af94586-c203-44ef-b704-2fd7d5106a0d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:11.008512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 821/1647] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:11.008512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:11.008512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:11.118113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:11.119655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:11.134585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171584ca-b124-4438-8df1-ac8ba969ec9e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:11.135291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:11.135710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] PATH_INFO: `/v3/users/cf97b651a3e5475c9656c165a24f0d9a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:11.138764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:11.139591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:11.140059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:11.140378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:11.141428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf97b651a3e5475c9656c165a24f0d9a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:11.148585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:11.158931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:11.164276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:11.180779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 822/1648] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:11.182116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:11.204029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369cb68-05dd-4795-9a4e-f18aca5064ff service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:11.209811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:11.210559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:11.211028 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:11.211719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:11.223322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:11.225696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e85bd-e15c-43ac-8273-b0d29d327de6 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:11.240219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf97b651a3e5475c9656c165a24f0d9a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:11.240772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 823/1649] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:11.261221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf97b651a3e5475c9656c165a24f0d9a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:11.262092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf97b651a3e5475c9656c165a24f0d9a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:11.275683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf97b651a3e5475c9656c165a24f0d9a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:11.287336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf97b651a3e5475c9656c165a24f0d9a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:11.297022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 827/1650] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:10 2022] DELETE /identity/v3/users/cf97b651a3e5475c9656c165a24f0d9a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:11.301891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:11.302453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:11.302693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:11.325251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:11.327130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:11.400397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 824/1651] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:11.428398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:11.431934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:11.453739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:11.577081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c141d-e55a-44d9-916b-8c4b6b9b993d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:11.582633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:11.583939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:11.584272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:11.584606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:11.606930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:11.609385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:11.622726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 828/1652] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:13.528114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:13.530113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:13.578646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c3a75-fabc-4623-84d6-4b5125788b91 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:13.592875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:13.598406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:13.604543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:13.605118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] PATH_INFO: `/v3/projects/8c91238659234e0b8e174b52cd7dba54` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:13.607470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c91238659234e0b8e174b52cd7dba54)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:13.611858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 20:08:13.611858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jul 27 20:08:13.611858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: Jul 27 20:08:13.623727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:13.823884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:13.876652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c91238659234e0b8e174b52cd7dba54', 'request_id': 'req-42fb0935-9817-4b98-894b-6b3fb099a3b2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:13.897191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c91238659234e0b8e174b52cd7dba54', 'request_id': 'req-42fb0935-9817-4b98-894b-6b3fb099a3b2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:13.923378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c91238659234e0b8e174b52cd7dba54', 'request_id': 'req-42fb0935-9817-4b98-894b-6b3fb099a3b2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:13.961466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] The token cache is being invalidate because project 8c91238659234e0b8e174b52cd7dba54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:13.961764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:13.972091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:13.978971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 825/1653] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:13 2022] DELETE /identity/v3/projects/8c91238659234e0b8e174b52cd7dba54 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:13.979796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:13.992170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440450-2382-4769-b47f-76808df95a87 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:14.147202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:14.147957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:14.148238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:14.148521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] PATH_INFO: `/v3/users/1cd0de6895334360914d96056b608c0f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:14.149784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd0de6895334360914d96056b608c0f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:14.152258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:14.267020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd0de6895334360914d96056b608c0f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.293178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd0de6895334360914d96056b608c0f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.294144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd0de6895334360914d96056b608c0f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.304286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd0de6895334360914d96056b608c0f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.339775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd0de6895334360914d96056b608c0f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.358303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 829/1654] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:13 2022] DELETE /identity/v3/users/1cd0de6895334360914d96056b608c0f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:14.470252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:14.471748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:14.516021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0935-9817-4b98-894b-6b3fb099a3b2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:14.598095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:14.600597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:14.686866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:14.687662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:14.688070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:14.688415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] PATH_INFO: `/v3/projects/730a466023454452a63effcf2672e10c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:14.691106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=730a466023454452a63effcf2672e10c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:14.695288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:14.730005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:14.803977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de0c0c-7c28-4233-a1f1-5e20e54c5636 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:14.824776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730a466023454452a63effcf2672e10c', 'request_id': 'req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.854095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730a466023454452a63effcf2672e10c', 'request_id': 'req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.870312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730a466023454452a63effcf2672e10c', 'request_id': 'req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.898062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] The token cache is being invalidate because project 730a466023454452a63effcf2672e10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:14.901267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.902073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3b2cbcad-a1a5-42fc-b612-f7c08bc5e621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:14.905276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 826/1655] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:14 2022] DELETE /identity/v3/projects/730a466023454452a63effcf2672e10c => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:14.910482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6e7d-11ed-4387-86c8-cb74fbcf3062 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:14.910853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6e7d-11ed-4387-86c8-cb74fbcf3062 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:14.911905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6e7d-11ed-4387-86c8-cb74fbcf3062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:14.964678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-32ae6e7d-11ed-4387-86c8-cb74fbcf3062 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:14.968755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-32ae6e7d-11ed-4387-86c8-cb74fbcf3062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:14.987807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:14.988496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:14.988783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:14.989090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] PATH_INFO: `/v3/projects/4fa623a3ef594b2bba664198d0326167` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:14.992512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa623a3ef594b2bba664198d0326167)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:14.999985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:15.128917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:15.156087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa623a3ef594b2bba664198d0326167', 'request_id': 'req-0c70558a-ab4f-4bc8-96c2-34a3460157dd'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.159954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 827/1656] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:15.167894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:15.168374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:15.168831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:15.170146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa623a3ef594b2bba664198d0326167', 'request_id': 'req-0c70558a-ab4f-4bc8-96c2-34a3460157dd'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.184644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa623a3ef594b2bba664198d0326167', 'request_id': 'req-0c70558a-ab4f-4bc8-96c2-34a3460157dd'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.210450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] The token cache is being invalidate because project 4fa623a3ef594b2bba664198d0326167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:15.210745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.213883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.220707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 830/1657] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:14 2022] DELETE /identity/v3/projects/4fa623a3ef594b2bba664198d0326167 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:15.222236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:15.226077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:15.246133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:15.249800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:15.374653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70558a-ab4f-4bc8-96c2-34a3460157dd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:15.390350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:15.391074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:15.391423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:15.391623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] PATH_INFO: `/v3/users/beab9378ea824814b9f57aa403fac44d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:15.393244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beab9378ea824814b9f57aa403fac44d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:15.397724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:15.433369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:15.435837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 828/1658] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:15.445908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:15.477693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079b4f9-2730-4409-a67b-0921ab92bdde None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:15.501892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:15.501892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:15.502350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:15.502518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:15.503191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:15.503499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:15.505235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:15.510961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beab9378ea824814b9f57aa403fac44d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.535322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 829/1659] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:15.536945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:15.546519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beab9378ea824814b9f57aa403fac44d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.548711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beab9378ea824814b9f57aa403fac44d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.566435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beab9378ea824814b9f57aa403fac44d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.566633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef1e24-5192-49d2-bba7-3a7c4cc68c1b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:15.574260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:15.574899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:15.575187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:15.575472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:15.576148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:15.578135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:15.585642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beab9378ea824814b9f57aa403fac44d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:15.601642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 831/1660] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:15 2022] DELETE /identity/v3/users/beab9378ea824814b9f57aa403fac44d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:15.640316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 830/1661] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:15 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:15.641038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:15.642569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:15.791741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb535c-f1a6-4951-9eb9-8793afdc1597 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:15.879085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:15.880771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:15.881091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:15.881369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:15.883625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:15.886004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:15.931850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 832/1662] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:15 2022] POST /identity/v3/users => generated 282 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:15.938218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:15.971203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95606fc5-b601-4804-b2c5-ab933c9976d2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:15.989266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.004792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.006858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.007164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.009381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.015162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.056367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.068593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 831/1663] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:16.083033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c92e87-632f-4110-9d6f-3187899a208f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.095019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.095671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.096060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.096348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.097433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.100287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db7fc1-a03e-4981-9ac8-fd26ce4f4300 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.121848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.128293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 833/1664] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:16.135826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b247bf-f926-408e-9415-bdbfd31b006b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.151447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.152302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.153226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.153568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] PATH_INFO: `/v3/projects/15819c00a197497f9d9bab00a8804239/users/80c24c5cedea4b6b958f308ce0f71dbe/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.196595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=15819c00a197497f9d9bab00a8804239, user_id=80c24c5cedea4b6b958f308ce0f71dbe, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.200676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.243340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 832/1665] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:16 2022] PUT /identity/v3/projects/15819c00a197497f9d9bab00a8804239/users/80c24c5cedea4b6b958f308ce0f71dbe/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:16.248361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.248840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.249196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.309194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.311585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:16.319407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] MFA Rules not processed for user `80c24c5cedea4b6b958f308ce0f71dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:16.321835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:16.332330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:16.465051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031384e1-d81f-4e46-831c-415c91f944cb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.471913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 834/1666] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:16.473183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.480837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.481568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.482437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.482437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] PATH_INFO: `/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.483464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8f8b7b6b7c4f4d8de21b3fe6ce37c1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.493234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.510460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218deef7-4f2a-4d82-931a-27c679e521af None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.530156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.531589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.531874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.532211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.540121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.543473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.588833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:16.618087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8f8b7b6b7c4f4d8de21b3fe6ce37c1', 'request_id': 'req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:16.632414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8f8b7b6b7c4f4d8de21b3fe6ce37c1', 'request_id': 'req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:16.641514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 835/1667] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:16.646359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.647930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8f8b7b6b7c4f4d8de21b3fe6ce37c1', 'request_id': 'req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:16.673306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] The token cache is being invalidate because project 3d8f8b7b6b7c4f4d8de21b3fe6ce37c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:16.673620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:16.676376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6ffcb1f9-a88d-4d26-98f5-73b6d3820ce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:16.681196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 833/1668] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:16 2022] DELETE /identity/v3/projects/3d8f8b7b6b7c4f4d8de21b3fe6ce37c1 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:16.688591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae9bd-60ec-40f7-8bed-a772a5f2f726 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.689542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae9bd-60ec-40f7-8bed-a772a5f2f726 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.690651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae9bd-60ec-40f7-8bed-a772a5f2f726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.728248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8b9ae9bd-60ec-40f7-8bed-a772a5f2f726 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:16.729768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8b9ae9bd-60ec-40f7-8bed-a772a5f2f726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:16.790234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407322c-f538-491a-a8e8-6395f98cdb8e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.806645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 834/1669] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:16.815860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.816411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.816941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.817439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.821842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.822304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.822758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.823626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.827478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.860923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:16.863525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:16.879103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 836/1670] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:16.886238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.891449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:16.897814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 835/1671] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:16.899484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.925475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1bc43-3326-4939-b8a2-e8c6cb5a44d5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.930602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e110b-9d70-43d5-9f45-3136db0d835a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:16.942119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.946431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:16.946802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.947167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.947522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.949705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:16.950044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:16.950422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:16.951133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:16.951383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.953147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.953865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:16.958465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:16.971069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 836/1672] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:16.975788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.984169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 837/1673] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:16.988120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:16.997624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bcb65-d877-40ca-bba7-cfd3737fb806 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.007698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03829aad-4cf5-47dc-a53d-d7ba1c96a24f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.010833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.012700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.013583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.014866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.016224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.019445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.020275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.021139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.024006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.024532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] PATH_INFO: `/v3/projects/8eae7271a12e4007a6d7599082caa49c/users/a6bde2bd87c64df7b84c69cc41787cfb/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.061263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eae7271a12e4007a6d7599082caa49c, user_id=a6bde2bd87c64df7b84c69cc41787cfb, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.070114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.081463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 837/1674] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:16 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:17.087501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.101908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 838/1675] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:16 2022] PUT /identity/v3/projects/8eae7271a12e4007a6d7599082caa49c/users/a6bde2bd87c64df7b84c69cc41787cfb/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:17.108850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.212622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:17.260550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5b09-3f78-4a76-9cd8-3537006710ed admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.261429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a8ef0-5947-43ed-a71e-b03240ecb12a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.281437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.282382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.284890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.285206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.285791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.288377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.288697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.289355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.291874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.293465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.298144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.298849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.343862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 839/1676] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:17.345958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.357999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 838/1677] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:17 2022] POST /identity/v3/users => generated 285 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:17.362046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.373824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7843e-9e1d-4978-8539-ff09a781fb17 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.392388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.393438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.393605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227554e-cc41-43a5-aa70-aef1e670fe3d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.394063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.394508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] PATH_INFO: `/v3/projects/8eae7271a12e4007a6d7599082caa49c/users/a6bde2bd87c64df7b84c69cc41787cfb/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.398368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eae7271a12e4007a6d7599082caa49c, user_id=a6bde2bd87c64df7b84c69cc41787cfb, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.401393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18236d2a-832b-48ed-a0b2-26cbb43ce0a0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.404752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.405989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.406282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.408176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.411906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.415373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.421676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:17.427644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 840/1678] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:17 2022] PUT /identity/v3/projects/8eae7271a12e4007a6d7599082caa49c/users/a6bde2bd87c64df7b84c69cc41787cfb/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:17.438618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.439656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.439656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.447665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 839/1679] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:17.448600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.470413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36618328-4087-45c1-af16-96c05acfe68e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.500189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] MFA Rules not processed for user `a6bde2bd87c64df7b84c69cc41787cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:17.501452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:17.634708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.637863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.638460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.639006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.640160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.642609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.663508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 840/1680] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:17.664203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 841/1681] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:17.667938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.678509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:17.679974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:17.714595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc2309-ba69-45bd-a164-e44210793fa2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.730034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.730984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.731390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.731821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] PATH_INFO: `/v3/projects/4bd26aa301e943fea53aed6432e0a43a/users/8c3987fa73924b169e3f51119604cf2a/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.771937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd26aa301e943fea53aed6432e0a43a, user_id=8c3987fa73924b169e3f51119604cf2a, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:17.775283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaf9a5-9145-4738-a458-35b321f11049 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:17.809464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 841/1682] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:17 2022] PUT /identity/v3/projects/4bd26aa301e943fea53aed6432e0a43a/users/8c3987fa73924b169e3f51119604cf2a/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:17.820475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdf8fe-f647-40e1-9c0c-ad7c1c5a8aa6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:17.821198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.822196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.823787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.900945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] MFA Rules not processed for user `8c3987fa73924b169e3f51119604cf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:17.904731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:17.931786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:17.932140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:17.932423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:17.932702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:17.971305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:18.075383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 842/1683] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:18.076878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:18.079815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:18.106414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.107786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.135160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 842/1684] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:18.145830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:18.261983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f6bc6-1f35-4758-8dc0-4782cd0a7e59 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:18.278275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:18.278960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.279243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.279519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] PATH_INFO: `/v3/projects/68bccff39d43455bb4846271ce0bebae` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.283627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68bccff39d43455bb4846271ce0bebae)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.289702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.291625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:18.337198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6389e9-fa22-41c4-bd29-0910894e3f6f service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:18.348586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:18.362872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.363605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.364573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.365895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.392057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.462217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68bccff39d43455bb4846271ce0bebae', 'request_id': 'req-04fded98-ab31-4d31-b55c-3df83ea1922f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:18.482491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68bccff39d43455bb4846271ce0bebae', 'request_id': 'req-04fded98-ab31-4d31-b55c-3df83ea1922f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:18.494007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 843/1685] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:18 2022] POST /identity/v3/projects => generated 369 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:18.495279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:18.498941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68bccff39d43455bb4846271ce0bebae', 'request_id': 'req-04fded98-ab31-4d31-b55c-3df83ea1922f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:18.526237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] The token cache is being invalidate because project 68bccff39d43455bb4846271ce0bebae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:18.527232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:18.528467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:18.533360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 843/1686] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:18 2022] DELETE /identity/v3/projects/68bccff39d43455bb4846271ce0bebae => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:18.536457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:18.539234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:18.636011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763726c-746b-4e0e-94a9-3eff3618e3fa admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:18.649661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:18.650361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.650643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.650922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.655322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:18.704416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fded98-ab31-4d31-b55c-3df83ea1922f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:18.780808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:18.793934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.796871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ace8-64b9-4528-813d-29e9b4204902 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.823782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 844/1687] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:18.825921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.826217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.826548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.852318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:18.854400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.854696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.855029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.855714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.862169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76547d-8613-4306-9591-4dd7c5ae1a97 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.882497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:18.886346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:18.915098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 845/1688] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:18.915271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 844/1689] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:18 2022] POST /identity/v3/users => generated 291 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:18.915372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:18.916811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.919816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.920107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.943027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f679b74-76f9-4e63-a9ee-75b3ba2c9b66 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:18.961526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:18.962153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:18.962437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:18.962714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:18.963422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:18.972018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:18.978111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:18.979370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:19.012486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 846/1690] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:19.013183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.023874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 845/1691] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:19.025091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.045744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af65ca30-2c65-4f1f-9419-c2aed6fdc7dd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.049543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39834307-8d6a-4643-b32f-07aa885e4604 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.061915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.062458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.062706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.063140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.063412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.063527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.063819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.064594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:19.064945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.065809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] PATH_INFO: `/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a/users/403cbee7e5b8410fbbd70f4142d1a183/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.067590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.105149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 846/1692] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:19.106239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.131205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b811-1536-4f48-bede-d882218d0e16 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.150125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bc9338b6ae4dca9d59c151f0bf0b7a, user_id=403cbee7e5b8410fbbd70f4142d1a183, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.153333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a312a0-cf96-4737-bfe2-9683c2a646b7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.163873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.164521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.164805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.165112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.165917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.169214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.175746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:19.188934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 847/1693] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:19 2022] PUT /identity/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a/users/403cbee7e5b8410fbbd70f4142d1a183/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:19.188934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.188934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.188934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.231534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 847/1694] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:19 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:19.231534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.268674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] MFA Rules not processed for user `b734a345bfe5483d9a5a1e35d89c935b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:19.271353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:19.393140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a719075-e837-4d5b-bf8a-be8bbe391b99 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.402717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.403496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.403831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.404280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.404821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.406500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.436553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 848/1695] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:19.436553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.455818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777379d0-55f3-4537-8239-0a124706032f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.468892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.469629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.469863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.470256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.470817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.472907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.497248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 848/1696] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:19.511889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.539669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89374b2-13c9-44e5-9c5f-e2b35eccd2e3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.556236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.556942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.557247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.559368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] PATH_INFO: `/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a/users/403cbee7e5b8410fbbd70f4142d1a183/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.559536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 849/1697] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:19 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:19.560538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.564995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bc9338b6ae4dca9d59c151f0bf0b7a, user_id=403cbee7e5b8410fbbd70f4142d1a183, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.566431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:19.571887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.601862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 849/1698] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:19 2022] PUT /identity/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a/users/403cbee7e5b8410fbbd70f4142d1a183/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:19.604627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:19.761165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ce302-c101-44fa-8e0c-aaea6867f4c7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.778531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:19.779045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.779334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.779604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.780327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:19.782444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fda87-ae52-4856-ac7d-c1b3b029f71a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:19.814996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 850/1699] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:19.819095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:19.820233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:19.820365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:19.845903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52556b2e-b8cb-4cdc-b721-70d65e93958a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:19.916924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] MFA Rules not processed for user `403cbee7e5b8410fbbd70f4142d1a183`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:19.925883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:19.981962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:20.033087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:20.091990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:20.095653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:20.096698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:20.097191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:20.186491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 851/1700] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:20.189718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:20.223907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d58b9e-41b3-4ec3-9ab5-2384c67d4050 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:20.236250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:20.237439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:20.237687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:20.241356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:20.245527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:20.250634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:20.292778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 852/1701] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:20.301755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:20.303413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:20.310218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:20.320371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:20.353218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 850/1702] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:20.354111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:20.392201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cea9c-3d51-4fb4-8ee9-228872063174 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:20.417604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:20.426942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:20.428057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:20.428368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] PATH_INFO: `/v3/projects/4570bbb74c0e489881f56f395225da25/users/a7b81d959ab64fcdb0d6258993cdc83d/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:20.506241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4570bbb74c0e489881f56f395225da25, user_id=a7b81d959ab64fcdb0d6258993cdc83d, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:20.520029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b178a-2835-4ddc-a046-bd1b5fc436f6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:20.572254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 851/1703] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:20 2022] PUT /identity/v3/projects/4570bbb74c0e489881f56f395225da25/users/a7b81d959ab64fcdb0d6258993cdc83d/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:20.584231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:20.584494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:20.585090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:20.589524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ab51c-b1c2-45e5-9cfe-752d5433f2e4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:20.672049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] MFA Rules not processed for user `a7b81d959ab64fcdb0d6258993cdc83d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:20.673666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:20.753530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:20.754319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:20.754744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:20.755808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:20.807536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 852/1704] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:20.812321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:20.933455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:20.935665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1904686-1274-467a-a6a0-8e3f0ca2ceb9 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:20.990766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 853/1705] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:20.998831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252e693-eea0-499d-bcb9-bd5b5906b0b5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.000124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39ee7-7bbb-4181-a5ae-7ed8b59f4a20 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.000418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39ee7-7bbb-4181-a5ae-7ed8b59f4a20 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.000693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39ee7-7bbb-4181-a5ae-7ed8b59f4a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.017265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:21.018279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.018569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.018878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.019516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:21.024197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:21.025318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:21.071291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:21.095255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4ed39ee7-7bbb-4181-a5ae-7ed8b59f4a20 None None] MFA Rules not processed for user `a48eaef4d70b4116bd307374d819bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:21.096753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ed39ee7-7bbb-4181-a5ae-7ed8b59f4a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:21.118200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 853/1706] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:20 2022] POST /identity/v3/projects => generated 365 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:21.120938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:21.287046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 854/1707] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:21.303070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.303364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.303644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.315348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56872d83-3995-4bc2-94ce-ffb7fe43a009 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.331418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:21.332595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.335525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.335825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.337572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:21.341545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:21.359518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] MFA Rules not processed for user `a48eaef4d70b4116bd307374d819bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:21.360816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:21.388224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 854/1708] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:21 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:21.391964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:21.426329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c55016-f3c5-4f94-a405-5e400c30994d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.433616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:21.434414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.434794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.435223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.436030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:21.438160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:21.453147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 855/1709] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:21.458519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:21.472124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf5faa-bc53-48ad-9fc2-d417693ab50d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.479962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:21.481418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.483239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.483490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] PATH_INFO: `/v3/projects/c93154ccbd844f60aa09e72251f1d602/users/755db3168ed34d47ae63e0d269deb947/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.514488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93154ccbd844f60aa09e72251f1d602, user_id=755db3168ed34d47ae63e0d269deb947, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:21.523228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:21.552393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 856/1710] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:21 2022] PUT /identity/v3/projects/c93154ccbd844f60aa09e72251f1d602/users/755db3168ed34d47ae63e0d269deb947/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:21.556352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:21.560620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 855/1711] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:21.573214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:21.762919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1908c-ff55-47ad-83fe-1ff591c8494a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.775620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:21.778445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.778950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.780144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.782321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:21.785805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499f735-2421-4564-8d3e-fb54b3518ec9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:21.813687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 857/1712] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:21.816161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:21.816637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:21.818383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:21.892549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e639284-b8d4-4597-b556-dd2036b640bb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:21.907456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] MFA Rules not processed for user `ea3c5c3007e04391b5b108473da62787`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:21.909191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:21.983685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:22.015027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:22.127047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 858/1713] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:22.128397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:22.170483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:22.171197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:22.171481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:22.171771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:22.172877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2c370-e1ce-4f46-8729-0c1b6afef9da None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:22.206037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:22.206750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:22.207034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:22.207314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] PATH_INFO: `/v3/projects/c93154ccbd844f60aa09e72251f1d602/users/755db3168ed34d47ae63e0d269deb947/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:22.215347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c93154ccbd844f60aa09e72251f1d602, user_id=755db3168ed34d47ae63e0d269deb947, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:22.218619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:22.256647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 859/1714] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:22 2022] PUT /identity/v3/projects/c93154ccbd844f60aa09e72251f1d602/users/755db3168ed34d47ae63e0d269deb947/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:22.257837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:22.263765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:22.361624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:22.363274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095827e-aaa9-4c36-a9a2-9b1f5f7b18c3 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:22.392247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 856/1715] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:22.394979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:22.395826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:22.396198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:22.467542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566483c0-5390-4ec8-965c-d78dfe54e6df admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:22.475345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] MFA Rules not processed for user `755db3168ed34d47ae63e0d269deb947`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:22.477606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:22.496397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:22.497074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:22.499537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:22.499825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:22.660643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 857/1716] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:22.662927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:22.666374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:22.729717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:22.731365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:22.758001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 860/1717] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:22.886374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeab832-3886-4d95-acb0-47e75710c14e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:22.904309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:22.905121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:22.906453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:22.906851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:22.930303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:22.931933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a55b8-5f80-45fb-a7af-80ba680769ae service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:22.959227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 858/1718] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:23.213156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:23.231421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfd6e1-9b88-454d-828c-3b80d77cac46 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:23.243963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:23.245050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:23.246966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:23.247564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:23.388483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:23.411634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:23.413698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3dccd-e0fa-47e7-a30d-3e80392838a1 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:23.436436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 861/1719] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:23.644025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:23.644540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:23.645021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:23.655508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:23.716617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] MFA Rules not processed for user `ec4599fb9568454ba36b01038c345248`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:23.727621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:23.875371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 859/1720] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:23.886941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:23.887368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:23.887557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:23.931769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] MFA Rules not processed for user `ec4599fb9568454ba36b01038c345248`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:23.935944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:23.958811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 862/1721] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:23.969672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:23.992497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f72e1-2a0a-4ef9-8ac7-fbb6d3a9abbf None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:23.997384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:23.998814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:24.014154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:24.015870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.016271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.016998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.042792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:24.051777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c5045-6ee7-4996-85dd-462ad51e9cd3 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:24.115515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 860/1722] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:24.118683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47889-6495-405c-af51-d3e40b5224af None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.120710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47889-6495-405c-af51-d3e40b5224af None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.122965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d47889-6495-405c-af51-d3e40b5224af None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.131473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 861/1723] 10.222.0.65 () {54 vars in 912 bytes} [Wed Jul 27 20:08:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:24.135953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b657-c341-4741-9237-a9af52bced36 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.140899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b657-c341-4741-9237-a9af52bced36 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.142922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b657-c341-4741-9237-a9af52bced36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.253839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba44d0-3edf-46ed-84d2-d585710187a5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:24.275634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:24.276319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.276605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.277179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] PATH_INFO: `/v3/users/80c24c5cedea4b6b958f308ce0f71dbe` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.281077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c24c5cedea4b6b958f308ce0f71dbe)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:24.283445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:24.318441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9261b657-c341-4741-9237-a9af52bced36 None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:24.350620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9261b657-c341-4741-9237-a9af52bced36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:24.417544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c24c5cedea4b6b958f308ce0f71dbe'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:24.431482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c24c5cedea4b6b958f308ce0f71dbe'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:24.471771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c24c5cedea4b6b958f308ce0f71dbe'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:24.499617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c24c5cedea4b6b958f308ce0f71dbe'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:24.500944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-779d9677-943c-4825-ac8a-848a663ae786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c24c5cedea4b6b958f308ce0f71dbe'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:24.515461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 863/1724] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:23 2022] DELETE /identity/v3/users/80c24c5cedea4b6b958f308ce0f71dbe => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:24.517683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda7a1-05c6-430f-8a23-3cd6bfa4e0a3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.518100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda7a1-05c6-430f-8a23-3cd6bfa4e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.518714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bda7a1-05c6-430f-8a23-3cd6bfa4e0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.531620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 862/1725] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:24.545267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-81bda7a1-05c6-430f-8a23-3cd6bfa4e0a3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:24.559716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-81bda7a1-05c6-430f-8a23-3cd6bfa4e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:24.563373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b657-c341-4741-9237-a9af52bced36 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:24.565734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9261b657-c341-4741-9237-a9af52bced36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:24.583434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b657-c341-4741-9237-a9af52bced36 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:24.731209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:24.731896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.732115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.732394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.741766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:24.743580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:24.764741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 863/1726] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:24.774564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 864/1727] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:24.838542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:24.854191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2616d-8ec6-4b9d-9895-44ce027e1e64 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:24.868895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:24.869656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:24.869947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:24.870230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:24.889667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:24.891278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f094c5-35f4-4b23-bff2-a87feea35b2f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:24.925016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 864/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:25.182711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:25.183668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:25.184119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:25.185670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 865/1729] 10.222.0.65 () {54 vars in 912 bytes} [Wed Jul 27 20:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:25.192237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:25.193036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:25.193867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:25.210620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:25.245395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:25.247342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:25.251277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:25.252526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:25.274954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 865/1730] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:08:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:25.303332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:25.326043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959e63d-6f0b-4b1c-9228-0f0e877c7c19 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:25.355783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:25.357188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:25.358189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:25.358719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] PATH_INFO: `/v3/users/8c3987fa73924b169e3f51119604cf2a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:25.362681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3987fa73924b169e3f51119604cf2a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:25.368514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:25.369880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:25.370098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:25.388280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117867eb-74d0-439f-b40c-182cb4159f6f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:25.405300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:25.405918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:25.406206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:25.416708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:25.458499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:25.460267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a026262-89f6-41b7-8976-1c0bb19c1784 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:25.540877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 866/1731] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:25.546555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3987fa73924b169e3f51119604cf2a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:25.565841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3987fa73924b169e3f51119604cf2a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:25.595429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3987fa73924b169e3f51119604cf2a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:25.613387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3987fa73924b169e3f51119604cf2a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:25.615513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3987fa73924b169e3f51119604cf2a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:25.630318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 866/1732] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:25 2022] DELETE /identity/v3/users/8c3987fa73924b169e3f51119604cf2a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:25.635708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c77e-05a5-4200-852a-17be56b64b17 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:25.636233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c77e-05a5-4200-852a-17be56b64b17 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:25.636669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65c77e-05a5-4200-852a-17be56b64b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:25.690722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2c65c77e-05a5-4200-852a-17be56b64b17 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:25.695541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2c65c77e-05a5-4200-852a-17be56b64b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:25.908196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 867/1733] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:25.953411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:25.955086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:26.223767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbb8ed-00a7-4e23-8612-623ba1c1218c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:26.252152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:26.253861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:26.254158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:26.254976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:26.286600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:26.288330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b49592-b8dc-4402-8ac2-7376073e3548 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:26.332594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 867/1734] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:27.096865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada68c-307a-49e1-85fb-e1b3f01e813f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.097719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada68c-307a-49e1-85fb-e1b3f01e813f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.098680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ada68c-307a-49e1-85fb-e1b3f01e813f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.100814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 868/1735] 10.222.0.65 () {54 vars in 961 bytes} [Wed Jul 27 20:08:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:27.108490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.108994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.109372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.157284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:27.162978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:27.420872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 868/1736] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Jul 27 20:08:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:27.430452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ada68c-307a-49e1-85fb-e1b3f01e813f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:27.478997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:27.480157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ada68c-307a-49e1-85fb-e1b3f01e813f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:27.482568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:27.511227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:27.511881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.512163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.512453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.515963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc348b-956e-402a-aade-45ac6169f11a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:27.542597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:27.571094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:27.572186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.572534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.572819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.611014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:27.615474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:27.632003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:27.642731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 869/1737] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:27.646101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be32ad-e127-43b7-9716-1e9ed432b49c service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:27.676064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 869/1738] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:27.723702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:27.728473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:27.748583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.749655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.750013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.795224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:27.813963 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1f9b-69b4-4b3e-a6f1-1ef4e5d75b24 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:27.818841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] MFA Rules not processed for user `a48eaef4d70b4116bd307374d819bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:27.820411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:27.826599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:27.827281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:27.827565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:27.828197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] PATH_INFO: `/v3/projects/15819c00a197497f9d9bab00a8804239` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:27.831036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=15819c00a197497f9d9bab00a8804239)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:27.833753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:27.950244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15819c00a197497f9d9bab00a8804239', 'request_id': 'req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:27.980224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15819c00a197497f9d9bab00a8804239', 'request_id': 'req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:27.987920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15819c00a197497f9d9bab00a8804239', 'request_id': 'req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.014576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] The token cache is being invalidate because project 15819c00a197497f9d9bab00a8804239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:28.014867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.016352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.023533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 870/1739] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:28.026973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:28.029424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 870/1740] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:27 2022] DELETE /identity/v3/projects/15819c00a197497f9d9bab00a8804239 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:28.033428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:28.033708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:28.037120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:28.078468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646a7fa-1d95-4810-b54d-96a4c5c6d163 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:28.192177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd7503-c9d0-41dd-9e9c-9ac6c945c3db admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:28.212607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:28.213189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:28.214815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:28.225617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:28.226106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] PATH_INFO: `/v3/users/f851d33b4920425ca3f7e2eec8e72999` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:28.226417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:28.226895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:28.227808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] PATH_INFO: `/v3/users/a6bde2bd87c64df7b84c69cc41787cfb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:28.234250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bde2bd87c64df7b84c69cc41787cfb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:28.234858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f851d33b4920425ca3f7e2eec8e72999)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:28.237087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:28.239813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:28.298306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:28.368693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f851d33b4920425ca3f7e2eec8e72999'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.375059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bde2bd87c64df7b84c69cc41787cfb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.393880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bde2bd87c64df7b84c69cc41787cfb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.402437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f851d33b4920425ca3f7e2eec8e72999'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.403852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f851d33b4920425ca3f7e2eec8e72999'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.424486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f851d33b4920425ca3f7e2eec8e72999'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.447110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-90e7fb8d-734a-45c1-aac9-f97d48251725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f851d33b4920425ca3f7e2eec8e72999'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.451968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bde2bd87c64df7b84c69cc41787cfb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.470325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 871/1741] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:28 2022] DELETE /identity/v3/users/f851d33b4920425ca3f7e2eec8e72999 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:28.479715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:28.480175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:28.480628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:28.490039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bde2bd87c64df7b84c69cc41787cfb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.497258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-09fa109c-f78e-4ca5-9743-505391a46a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bde2bd87c64df7b84c69cc41787cfb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:28.518318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 871/1742] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:28 2022] DELETE /identity/v3/users/a6bde2bd87c64df7b84c69cc41787cfb => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:28.525491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:28.528787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:28.529360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:28.534021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] MFA Rules not processed for user `b734a345bfe5483d9a5a1e35d89c935b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:28.537559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:28.548418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:28.606347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:28.615219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:28.777489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 872/1743] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:28.872700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 872/1744] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:28.893096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:28.894731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:29.055456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b19f2d-f254-4482-b6f0-9ff27ab2a1e8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:29.075330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:29.077029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:29.089779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:29.089779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:29.117615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:29.124980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:29.187532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 873/1745] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:29.639391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:29.643518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:29.668534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:29.670480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:29.694710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f195cb-3eb4-449f-aeb6-5a8fae8d266d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:29.697890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6171e6a-83d5-44e8-b8f8-e9f73115ac58 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:29.714375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:29.715922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:29.715922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:29.716085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] PATH_INFO: `/v3/projects/8eae7271a12e4007a6d7599082caa49c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:29.721312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eae7271a12e4007a6d7599082caa49c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:29.724346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:29.733206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:29.733932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:29.735750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:29.736350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] PATH_INFO: `/v3/projects/4bd26aa301e943fea53aed6432e0a43a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:29.740058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd26aa301e943fea53aed6432e0a43a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:29.745717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:29.893222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:29.929329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd26aa301e943fea53aed6432e0a43a', 'request_id': 'req-f59047e1-c9b7-4b39-9bac-58ef302ab39d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:29.929477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eae7271a12e4007a6d7599082caa49c', 'request_id': 'req-e5ba9dd7-308d-4995-8f01-6f20509a253f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:29.941784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd26aa301e943fea53aed6432e0a43a', 'request_id': 'req-f59047e1-c9b7-4b39-9bac-58ef302ab39d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:29.964319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd26aa301e943fea53aed6432e0a43a', 'request_id': 'req-f59047e1-c9b7-4b39-9bac-58ef302ab39d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:29.964830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eae7271a12e4007a6d7599082caa49c', 'request_id': 'req-e5ba9dd7-308d-4995-8f01-6f20509a253f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:29.990872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eae7271a12e4007a6d7599082caa49c', 'request_id': 'req-e5ba9dd7-308d-4995-8f01-6f20509a253f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.019236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] The token cache is being invalidate because project 8eae7271a12e4007a6d7599082caa49c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:30.019542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.021958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e5ba9dd7-308d-4995-8f01-6f20509a253f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.031890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 874/1746] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:29 2022] DELETE /identity/v3/projects/8eae7271a12e4007a6d7599082caa49c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:30.033406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4f52-69a0-48dc-85a9-47217c74eef2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.033706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4f52-69a0-48dc-85a9-47217c74eef2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.033985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4f52-69a0-48dc-85a9-47217c74eef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.063641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] The token cache is being invalidate because project 4bd26aa301e943fea53aed6432e0a43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:30.065885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.074889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9a6f4f52-69a0-48dc-85a9-47217c74eef2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:30.076200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9a6f4f52-69a0-48dc-85a9-47217c74eef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.077049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.083892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 873/1747] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:29 2022] DELETE /identity/v3/projects/4bd26aa301e943fea53aed6432e0a43a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:30.086661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:30.097587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.210097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 875/1748] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:30.211932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.212386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.213428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.302921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:30.310700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.373452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 876/1749] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:30.377574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:30.382155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59047e1-c9b7-4b39-9bac-58ef302ab39d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:30.388254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:30.389903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.391104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.391500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] PATH_INFO: `/v3/users/403cbee7e5b8410fbbd70f4142d1a183` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.393967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=403cbee7e5b8410fbbd70f4142d1a183)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:30.398631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:30.419106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e8a6a-cee3-4390-bb4e-95571bae8573 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:30.436255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:30.438206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.449762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.450558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.453707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:30.454216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:30.456274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:30.499949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 877/1750] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:08:30.501110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:30.501736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:30.503283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.521333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cad07-bf72-4a1b-9d8f-2cb745e32a06 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:30.529322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:30.530320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.530746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.531330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] PATH_INFO: `/v3/projects/68afb3e7ea43454fa92d2d21560432f2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.532676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68afb3e7ea43454fa92d2d21560432f2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:30.535065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:30.554458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '403cbee7e5b8410fbbd70f4142d1a183'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.573800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '403cbee7e5b8410fbbd70f4142d1a183'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.618909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '403cbee7e5b8410fbbd70f4142d1a183'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.639238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68afb3e7ea43454fa92d2d21560432f2', 'request_id': 'req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.641879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '403cbee7e5b8410fbbd70f4142d1a183'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.644356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0d89b2f5-1653-4d1b-89fc-0933f6aeb744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '403cbee7e5b8410fbbd70f4142d1a183'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.654853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68afb3e7ea43454fa92d2d21560432f2', 'request_id': 'req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.659842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 874/1751] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:30 2022] DELETE /identity/v3/users/403cbee7e5b8410fbbd70f4142d1a183 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:30.659842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fafb-bc7c-4105-99c7-c60b3f7f9eea None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.659842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fafb-bc7c-4105-99c7-c60b3f7f9eea None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.661289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718fafb-bc7c-4105-99c7-c60b3f7f9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.677422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68afb3e7ea43454fa92d2d21560432f2', 'request_id': 'req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.701555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] The token cache is being invalidate because project 68afb3e7ea43454fa92d2d21560432f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:30.701851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.704952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2718fafb-bc7c-4105-99c7-c60b3f7f9eea None None] MFA Rules not processed for user `ea3c5c3007e04391b5b108473da62787`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:30.706598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:30.707846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2718fafb-bc7c-4105-99c7-c60b3f7f9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.708739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 878/1752] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:30 2022] DELETE /identity/v3/projects/68afb3e7ea43454fa92d2d21560432f2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:30.713025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:30.717969 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:30.802343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:30.840584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 875/1753] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:30.842300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcfd4-70ac-4768-95ac-243bb922b1ee None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:30.842300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcfd4-70ac-4768-95ac-243bb922b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:30.842574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abcfd4-70ac-4768-95ac-243bb922b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:30.888329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb05efa-bf23-4fa4-b2ce-b4cc7f988163 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:30.898664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e2abcfd4-70ac-4768-95ac-243bb922b1ee None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:30.900345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e2abcfd4-70ac-4768-95ac-243bb922b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:31.027120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.027995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.028288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.028568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.029456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.031563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.062133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 876/1754] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:31.072659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.075016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.075338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.128894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:31.130800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:31.134593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 879/1755] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:30 2022] POST /identity/v3/projects => generated 369 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:31.135789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.306314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6104-edf7-4d6c-ab30-f974e40a37ac admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.323096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.326873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.327166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.328840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.329994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.330977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:31.331982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.507065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 877/1756] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:31.515755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.530601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 880/1757] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:31 2022] POST /identity/v3/users => generated 285 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:31.532122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.574169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301dc77-25e5-42bc-98ce-a13b89f8d763 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.592950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c5718-ee85-4030-8482-5cb4e24bbded admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.597447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.598211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.598632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.599050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.600056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:31.600370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.607108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.608893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.609631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.610118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.611552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.613870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.626596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.647386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 881/1758] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:31.647386 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.678327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 878/1759] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:31.682929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453d1ff-23bd-44a9-8af8-a8beefb248a6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.686980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.713808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.715794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.718727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.719570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.720498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.726329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.735066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8123c-b110-4853-afd7-745d8e2e4a5c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.747161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.748076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.749414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.750006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.751431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.751816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 882/1760] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:31.752836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.755347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.759751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:31.790317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b569365-60c1-4a98-9430-c40c878c4572 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.796688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.797919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.798497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.799017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] PATH_INFO: `/v3/users/a7b81d959ab64fcdb0d6258993cdc83d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.801084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b81d959ab64fcdb0d6258993cdc83d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.805296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:31.809805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 879/1761] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:31 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:31.809805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:31.872180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:31.913472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045afc2-abc4-4b9a-b0bd-7bf8a78d9a03 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:31.927197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:31.929161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:31.929549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:31.930143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] PATH_INFO: `/v3/projects/63f9538c08c44912974abc080c389236/users/6cdfa792e30e4527bf6f2006e00ab2fa/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:31.933381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b81d959ab64fcdb0d6258993cdc83d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:31.971219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63f9538c08c44912974abc080c389236, user_id=6cdfa792e30e4527bf6f2006e00ab2fa, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:31.980633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:32.015429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b81d959ab64fcdb0d6258993cdc83d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.016878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 880/1762] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:31 2022] PUT /identity/v3/projects/63f9538c08c44912974abc080c389236/users/6cdfa792e30e4527bf6f2006e00ab2fa/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:32.017561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:32.018809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b81d959ab64fcdb0d6258993cdc83d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.019215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.038185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b81d959ab64fcdb0d6258993cdc83d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.080212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b81d959ab64fcdb0d6258993cdc83d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.104368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 883/1763] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:31 2022] DELETE /identity/v3/users/a7b81d959ab64fcdb0d6258993cdc83d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:32.104368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:32.105365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.229447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d3f1-d472-4b5c-8416-77679b1d34a4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:32.236588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdd616-46ee-43ec-87bd-d79d6b70b256 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:32.244732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:32.245799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.246043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:32.246193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.247055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.248337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.248915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] PATH_INFO: `/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.251705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bc9338b6ae4dca9d59c151f0bf0b7a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:32.253767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.255432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:32.268454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:32.275994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df7117-6d4f-4016-84e9-7260bab3a75a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:32.360894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bc9338b6ae4dca9d59c151f0bf0b7a', 'request_id': 'req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.375435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 881/1764] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:32 2022] POST /identity/v3/users => generated 286 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:32.378734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bc9338b6ae4dca9d59c151f0bf0b7a', 'request_id': 'req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.380991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144e7a-7098-4ec8-871b-59d8b0d2d731 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.381289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144e7a-7098-4ec8-871b-59d8b0d2d731 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.381576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19144e7a-7098-4ec8-871b-59d8b0d2d731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.396602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bc9338b6ae4dca9d59c151f0bf0b7a', 'request_id': 'req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.432212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] The token cache is being invalidate because project b9bc9338b6ae4dca9d59c151f0bf0b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:32.432562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.434554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-db4f1eb6-5a80-4aa1-9d50-f221d475fa00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:32.436714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 884/1765] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:32 2022] DELETE /identity/v3/projects/b9bc9338b6ae4dca9d59c151f0bf0b7a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:32.442030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.442458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.442750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.500629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:32.505684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.514352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-19144e7a-7098-4ec8-871b-59d8b0d2d731 None None] MFA Rules not processed for user `6cdfa792e30e4527bf6f2006e00ab2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:32.516440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-19144e7a-7098-4ec8-871b-59d8b0d2d731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.597691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:32.615451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:32.684333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 885/1766] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:32.685095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:32.692231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.721430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b809c7-d5a9-4476-9421-8afd1f044ad7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:32.733975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:32.734906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.735864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.736367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.738077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:32.741966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d49587-1e9b-4e91-9730-b5a798dfa6ea admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:32.767320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 886/1767] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:32.771680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.772144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.772583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.826394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:32.828729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:32.862009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 882/1768] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:32.867360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f1b9-4c9a-47ae-bf0f-23e2baceac07 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:32.867670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f1b9-4c9a-47ae-bf0f-23e2baceac07 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:32.868301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f1b9-4c9a-47ae-bf0f-23e2baceac07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:32.881206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 887/1769] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:32.882718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:32.888698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.061613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6e01f1b9-4c9a-47ae-bf0f-23e2baceac07 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:33.071109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e01f1b9-4c9a-47ae-bf0f-23e2baceac07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.134247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4eaec-ad9d-48db-b854-7653eae58b2e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.138601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.139720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.140003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.140379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.166836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.168855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.183756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 888/1770] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:33.184694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.199492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4631-a40f-4c9f-8eff-e880be5a9efc service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.205304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.205932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.206217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.206495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.207484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.211130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.224184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 889/1771] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:33.225129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.226902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.258723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794981d2-a89e-4304-a69f-cc9be8ae9fae admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.265426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 883/1772] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:08:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:33.266911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.267362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.267791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.288828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.292751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.293108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.294557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.296928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.300935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.340652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:33.342138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.352419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 884/1773] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:33.353723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.370947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e484cfa-8d4e-4fae-8205-88e0716b40f1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.377925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 890/1774] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:33 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:33.378902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.394261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.394992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.395305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.395556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] PATH_INFO: `/v3/projects/f0092f02cc104f75a37174a216e728a2/users/215cb37035f8416e892dae268e4f93bc/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.418555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0092f02cc104f75a37174a216e728a2, user_id=215cb37035f8416e892dae268e4f93bc, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.424998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.505059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 885/1775] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:33 2022] PUT /identity/v3/projects/f0092f02cc104f75a37174a216e728a2/users/215cb37035f8416e892dae268e4f93bc/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:33.506322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.554071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa45d-f33e-4363-9639-bd8b146db9d4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.648181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e5b6c-e6de-4438-bc67-dc8bc36416a1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.654798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.655705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.656046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.656467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.657399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.660345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:33.662587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.675264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:33.696607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 886/1776] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:33 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:33.696607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:33.698705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.713996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.715503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.716110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.716611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.717988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:33.718416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:33.720408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc0d50b-47b2-4ccd-817f-11ecab47dd8f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:33.738732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 891/1777] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:33.740912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.741330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.741724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:33.806653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] MFA Rules not processed for user `215cb37035f8416e892dae268e4f93bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:33.810237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:33.877263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005c5cd-05a7-4823-a3fb-b3ed5a959e6d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:33.886175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:33.887084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:33.887508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:33.887934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.047756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 892/1778] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:34.048829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.054044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.055676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:34.085572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 887/1779] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:08:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:08:34.086554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.132747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5558be-9e41-4d44-885c-57d904c5fdf2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.134292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494309f0-246b-4247-a47a-7b337f6b1efe service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.146870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:34.148169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:34.148169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:34.148169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.148734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.155368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:34.155787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:34.164069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:34.165934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:34.167767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.172777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.179987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:34.228742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 888/1780] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:34 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:34.228742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.238605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 893/1781] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:34.240468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.306952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256288a8-f6f4-4192-a55b-5b96d6689693 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.387752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:34.416316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f125f-1148-4a83-8c33-49da9cc27710 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.430241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:34.431321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:34.431600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:34.431882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.436002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.448021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:34.587917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 889/1782] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:34 2022] POST /identity/v3/projects => generated 373 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:34.589131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.625658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:34.631790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:34.632428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:34.633172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.635154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.642899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:34.728110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 894/1783] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:34 2022] POST /identity/v3/users => generated 293 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:34.728110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:34.769165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62395aaf-6d71-4fc9-bf27-ce5fc7502fc1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.856064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117ac6e-0987-4d62-8130-ff700425bf2f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:34.878320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:34.879811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:34.891613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:34.891910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] PATH_INFO: `/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98/users/f3676361044e47c589bf2906f58aac8e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:34.936619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:34.951768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b3f8cdc1b04519b78f32f8184fdf98, user_id=f3676361044e47c589bf2906f58aac8e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:34.967886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.009120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 890/1784] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:34 2022] PUT /identity/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98/users/f3676361044e47c589bf2906f58aac8e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:35.010612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.017917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.018451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.018837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.019116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.019817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.031405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.107673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932393ae-5252-41ba-bf82-c5883219dea4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.112544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.114011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.114501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.114892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.118568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 895/1785] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:34 2022] POST /identity/v3/users => generated 293 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:35.118825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.119518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.120655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.136702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 891/1786] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:35.136702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.151687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a3581-7b18-4621-8cee-393fd3518f47 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.153840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.154444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.154735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.155017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.155736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.158044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.173630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 892/1787] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:35.174606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.175861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beb2cf-53de-448e-9d63-2dd1616168f5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.186633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded1a-b04f-426d-ab6b-18d6d376fb44 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.191801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.192441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.192725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.193523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.194003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.195631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.215243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 893/1788] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:35.215446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.215446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.215446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.215446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.215739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.215811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.221432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.239947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cbc7f-2f41-46dc-9837-64f7cf0b5eac admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.253645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.254467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.254810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.255038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] PATH_INFO: `/v3/projects/445fc7fd0ae14328b6a014d8489d7400/users/730b179518bd48ebbf0ce689b942b406/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.287691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 896/1789] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:35.291899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445fc7fd0ae14328b6a014d8489d7400, user_id=730b179518bd48ebbf0ce689b942b406, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.293277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.301409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.324233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 894/1790] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:35 2022] PUT /identity/v3/projects/445fc7fd0ae14328b6a014d8489d7400/users/730b179518bd48ebbf0ce689b942b406/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:35.328010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.354019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8651b-f8fe-4167-9f5c-f2e6c530836f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.424085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:35.639515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d380641-7a00-4117-9e11-880acff207f9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.647911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.648609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.648892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.652064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] PATH_INFO: `/v3/projects/c4a3862408814673a6a5c6b52ec2c55f/users/abf1ae84c7ba4e5b907f82d57f03b5a0/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.690685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a3862408814673a6a5c6b52ec2c55f, user_id=abf1ae84c7ba4e5b907f82d57f03b5a0, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.691703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:35.697249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.700393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.703098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.704362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] PATH_INFO: `/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98/users/f3676361044e47c589bf2906f58aac8e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.715520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b3f8cdc1b04519b78f32f8184fdf98, user_id=f3676361044e47c589bf2906f58aac8e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:35.726024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f89667-0786-4610-a165-3539c4f1669a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:35.732579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 897/1791] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:35 2022] PUT /identity/v3/projects/c4a3862408814673a6a5c6b52ec2c55f/users/abf1ae84c7ba4e5b907f82d57f03b5a0/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:35.734288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:35.771730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 895/1792] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:35 2022] PUT /identity/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98/users/f3676361044e47c589bf2906f58aac8e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:35.775809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487723f4-0cdb-4711-b5e6-eace8a2f37c8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:35.777148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487723f4-0cdb-4711-b5e6-eace8a2f37c8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:35.779626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487723f4-0cdb-4711-b5e6-eace8a2f37c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:35.868348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc620e-cf3b-4e38-9d37-95932a676079 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:35.893935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-487723f4-0cdb-4711-b5e6-eace8a2f37c8 None None] MFA Rules not processed for user `abf1ae84c7ba4e5b907f82d57f03b5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:35.895895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-487723f4-0cdb-4711-b5e6-eace8a2f37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.036449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:36.110055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:36.110711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.110973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.111245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.111989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:36.117115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:36.140856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 898/1793] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:36.148822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:36.162323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.195898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 896/1794] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:36.197342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.197655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.197952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.201560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07415939-fac9-4d0c-bf44-905d57e59715 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:36.224548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:36.225268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.225553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.225835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] PATH_INFO: `/v3/projects/4570bbb74c0e489881f56f395225da25` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.229380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] RBAC: Authorizing `identity:delete_project(project_id=4570bbb74c0e489881f56f395225da25)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:36.231449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:36.297533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] MFA Rules not processed for user `f3676361044e47c589bf2906f58aac8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:36.301003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.363678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4570bbb74c0e489881f56f395225da25', 'request_id': 'req-ba048211-bc92-4253-9611-2c899c4781af'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:36.382082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4570bbb74c0e489881f56f395225da25', 'request_id': 'req-ba048211-bc92-4253-9611-2c899c4781af'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:36.421765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4570bbb74c0e489881f56f395225da25', 'request_id': 'req-ba048211-bc92-4253-9611-2c899c4781af'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:36.468497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] The token cache is being invalidate because project 4570bbb74c0e489881f56f395225da25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:36.468853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:36.472557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:36.478493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 899/1795] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:36 2022] DELETE /identity/v3/projects/4570bbb74c0e489881f56f395225da25 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:36.479288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:36.488404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.528141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 897/1796] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:36.529873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:36.531512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.564072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5993617-3773-4242-86a9-85fd419fcccd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:36.587312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:36.736195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:36.736853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.737183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.737466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.738152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:36.739075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba048211-bc92-4253-9611-2c899c4781af admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:36.742552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491a72a-7ef9-4872-83f1-a41dab10787c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:36.776808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:36.777550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.777831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.778107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] PATH_INFO: `/v3/projects/445fc7fd0ae14328b6a014d8489d7400/users/730b179518bd48ebbf0ce689b942b406/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.785355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] RBAC: Authorizing `identity:create_grant(project_id=445fc7fd0ae14328b6a014d8489d7400, user_id=730b179518bd48ebbf0ce689b942b406, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:36.788197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:36.793593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 898/1797] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:36 2022] POST /identity/v3/projects => generated 385 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:36.809579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:36.809750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:36.812717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:36.861422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 900/1798] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:36 2022] PUT /identity/v3/projects/445fc7fd0ae14328b6a014d8489d7400/users/730b179518bd48ebbf0ce689b942b406/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:36.864574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:36.872913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:36.883327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:36.884877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.081978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 899/1799] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 20:08:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:37.083668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.085630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.141383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:37.155291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b6cce-8c2d-4de5-88c3-73388596595d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.166641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.167769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.170294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.171326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] PATH_INFO: `/v3/users/755db3168ed34d47ae63e0d269deb947` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.173119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] RBAC: Authorizing `identity:delete_user(user_id=755db3168ed34d47ae63e0d269deb947)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.179174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.235531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07466659-a17b-4621-a6ae-7da6f31bdb8d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.240827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.241519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.241838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.242130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.323850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.325510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.337870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 900/1800] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:37.339187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.345434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:37.353785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755db3168ed34d47ae63e0d269deb947'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:37.354546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c0501-145c-4523-b6aa-80d2703b2f0a service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.362270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.362922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.363232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.363663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.364590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.366593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d2aa9-b7b2-41c5-84e5-9f802d07efb1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.401628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 901/1801] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:37 2022] POST /identity/v3/users => generated 299 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:37.403437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755db3168ed34d47ae63e0d269deb947'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:37.406182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755db3168ed34d47ae63e0d269deb947'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:37.407672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.408041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.408539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.425500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755db3168ed34d47ae63e0d269deb947'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:37.443504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755db3168ed34d47ae63e0d269deb947'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:37.467923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 901/1802] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:36 2022] DELETE /identity/v3/users/755db3168ed34d47ae63e0d269deb947 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:37.471219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.486056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.486277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] MFA Rules not processed for user `730b179518bd48ebbf0ce689b942b406`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:37.487520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.592237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 902/1803] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:37.598686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.602276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.654474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb8f99-2e12-42c1-9cd5-9cd28993b60b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.663723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.664381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.664665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.664938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.669004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.700059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.701440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.715806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 902/1804] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:37.717690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.721182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.724956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0688dd-6dbc-4733-ad6e-b736161ed6c6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.736036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.737170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.740573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.741932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.766729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.773801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.817601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 903/1805] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:37.818847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.825778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.834741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330f1c-8c56-490a-8c52-9cb310c48c2d service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:37.840778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:37.841474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:37.841757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:37.842068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:37.842756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:37.844400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:37.859499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 903/1806] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:37.860488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:37.862614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:37.893261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:38.003570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426ca7d-7baa-4438-b2b9-a9d25d72a8f9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.017288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.017942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.018228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.018508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.061695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.065369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:38.077880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef6d1d-c2fb-4e63-b0a2-164a74975f35 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.094943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.095905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.096200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.096537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.108984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:38.115604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 904/1807] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:38.116821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:38.165605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfebb4-7259-4a90-ba75-6e91b086731b service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.195637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.198302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.198629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.198904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] PATH_INFO: `/v3/projects/e26c8e1817784ea494c2ae0511cae3e9/users/ccaf6f5b45c2438a94c1a98f6a0bd177/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.207214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:38.264024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26c8e1817784ea494c2ae0511cae3e9, user_id=ccaf6f5b45c2438a94c1a98f6a0bd177, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.281951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:38.333165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 905/1808] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:38 2022] PUT /identity/v3/projects/e26c8e1817784ea494c2ae0511cae3e9/users/ccaf6f5b45c2438a94c1a98f6a0bd177/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:38.336299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:38.400506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.405541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:38.448798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 904/1809] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:38.450070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:38.493775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e390d0-6648-4ddf-96dd-a8504e3b1453 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.494149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd7f1-7422-43c5-8f3a-39cb6e737908 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.510425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.528712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.529101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.529390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.530163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.536996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:38.595393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 906/1810] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:38.595880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:38.639819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6d6fd-8c4d-4c76-8dad-56ff2f4d45ea admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:38.657249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.658053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.663399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.663698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] PATH_INFO: `/v3/projects/e26c8e1817784ea494c2ae0511cae3e9/users/ccaf6f5b45c2438a94c1a98f6a0bd177/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.669186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26c8e1817784ea494c2ae0511cae3e9, user_id=ccaf6f5b45c2438a94c1a98f6a0bd177, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.699906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb32569-cc77-4907-a8f2-5374f6a314ca admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:38.714649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:38.714649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.714649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.714649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.730811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:38.758977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 907/1811] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:38 2022] PUT /identity/v3/projects/e26c8e1817784ea494c2ae0511cae3e9/users/ccaf6f5b45c2438a94c1a98f6a0bd177/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:38.771227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:38.774197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:38.774481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:38.889895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] MFA Rules not processed for user `ccaf6f5b45c2438a94c1a98f6a0bd177`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:38.900647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:38.983483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:38.989395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:39.014262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 905/1812] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:39.126071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 908/1813] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:39.310909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:39.325055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0ea4-e531-4770-8e62-3c81e3e546e7 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:39.603845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:39.604671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:39.605185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:39.605658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:39.630291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:39.632791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ad65d-7a75-41c9-a756-9fd364d50b5a service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:39.664246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 906/1814] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:39.855583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:39.860732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:40.064234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781bd7b-d397-4206-bda3-71ab46d82ecd None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:40.087428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:40.090420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:40.093694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:40.101340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] PATH_INFO: `/v3/users/abf1ae84c7ba4e5b907f82d57f03b5a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:40.107761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf1ae84c7ba4e5b907f82d57f03b5a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:40.117979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:40.300029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf1ae84c7ba4e5b907f82d57f03b5a0'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:40.347869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf1ae84c7ba4e5b907f82d57f03b5a0'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:40.353117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf1ae84c7ba4e5b907f82d57f03b5a0'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:40.383727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf1ae84c7ba4e5b907f82d57f03b5a0'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:40.407197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf1ae84c7ba4e5b907f82d57f03b5a0'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:40.428130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 909/1815] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:39 2022] DELETE /identity/v3/users/abf1ae84c7ba4e5b907f82d57f03b5a0 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:40.429637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:40.430478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:40.430968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:40.497531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:40.514457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:40.588165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:40.728453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 907/1816] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:40.942181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:40.944288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:41.195149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48959df-5111-4eeb-ae3f-ce2db04b7573 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:41.215545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:41.216299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:41.216588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:41.216871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:41.253024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:41.255546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37345e61-9162-4422-890a-a473f55723e4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:41.306972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 910/1817] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:41.469007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:41.470597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:41.509731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809daca-9978-4549-9fa2-8cf088ac40a0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:41.526060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:41.527180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:41.527840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:41.529182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] PATH_INFO: `/v3/projects/c93154ccbd844f60aa09e72251f1d602` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:41.532518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c93154ccbd844f60aa09e72251f1d602)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:41.534632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:41.697664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c93154ccbd844f60aa09e72251f1d602', 'request_id': 'req-a095b066-cc0d-4f5b-b9c4-6756238ea223'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:41.733679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c93154ccbd844f60aa09e72251f1d602', 'request_id': 'req-a095b066-cc0d-4f5b-b9c4-6756238ea223'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:41.758508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c93154ccbd844f60aa09e72251f1d602', 'request_id': 'req-a095b066-cc0d-4f5b-b9c4-6756238ea223'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:41.810266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] The token cache is being invalidate because project c93154ccbd844f60aa09e72251f1d602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:41.810580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:41.811413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a095b066-cc0d-4f5b-b9c4-6756238ea223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:41.819486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 908/1818] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:41 2022] DELETE /identity/v3/projects/c93154ccbd844f60aa09e72251f1d602 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:41.819701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:41.819894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:41.820174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:41.872887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:41.874614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:41.928377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:42.047205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 911/1819] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:42.056597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.056893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.057231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.115240 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:42.118092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:42.149404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 909/1820] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:42.153864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.166184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5faf6b-f37f-4a15-8767-a16cf9295c99 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.174267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.175170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.175709 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.180354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.181809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:42.182277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.184107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.212852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.219698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 912/1821] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:42.225485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9196c33-3902-49e4-8bb1-35df5affaa94 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.231398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.232566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.233047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.233469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.234382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.236691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.278181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.288874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 910/1822] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:42 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:42.290262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dd785-c132-40cf-aed5-fadaba562e5d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.368690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.370846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.372906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.373361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.374219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.386503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.452317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 913/1823] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:42 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:42.456625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.526258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640ad3-5d0e-4532-8f23-c2565088c5ec admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.548799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.558496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.559647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.560951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.562942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.571106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.621425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 911/1824] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:42.625314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.673217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49687be0-3ecc-4237-a6db-352d9d92d486 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.708012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.708655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.708945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.709270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.710074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.711711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84737e-7090-4e6a-8174-740e9a3819b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.756336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 914/1825] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:42.757739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:42.774614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47099fb6-5b9d-4596-97e6-214107687b68 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:42.798050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:42.798929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.799371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.799814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] PATH_INFO: `/v3/projects/ef528c41502b4b13bb9f88a474503a01/users/e0a117fc71364c7894d0b5497ce04c89/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.842203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef528c41502b4b13bb9f88a474503a01, user_id=e0a117fc71364c7894d0b5497ce04c89, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:42.853004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:42.875037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 912/1826] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:42 2022] PUT /identity/v3/projects/ef528c41502b4b13bb9f88a474503a01/users/e0a117fc71364c7894d0b5497ce04c89/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:42.888158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:42.888712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:42.889188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:42.978672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] MFA Rules not processed for user `e0a117fc71364c7894d0b5497ce04c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:42.981405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:43.252827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 915/1827] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:43.256603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:43.438427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:43.536063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490d8c-1b85-4008-bdb4-5671875e7be2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:43.562268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:43.564291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:43.568593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:43.570061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:43.573608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:43.584741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:43.660939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 913/1828] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:43 2022] POST /identity/v3/projects => generated 387 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:43.670581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:43.718445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cf25e-c3ad-4aed-a8e3-c460cfb30a45 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:43.800345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:43.904562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:43.905229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:43.905509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:43.905780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:43.906453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:43.908651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:43.970693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 916/1829] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:43 2022] POST /identity/v3/users => generated 300 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:43.972069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.012367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa652ca-d81d-4d77-9a8a-8e96f61e0241 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.054633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:44.055785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.063331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.064117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.065259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:44.077922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:44.116504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.117856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 914/1830] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:44.130831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc4682-670d-40de-b059-455ae649ea19 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.148991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:44.149835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.157193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.157755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] PATH_INFO: `/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526/users/a48ce380d0d1451c82f6262f84461246/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.216829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a80fbc29cb4418a2c2b9975b9ff526, user_id=a48ce380d0d1451c82f6262f84461246, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:44.227309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:44.286110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 917/1831] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:44 2022] PUT /identity/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526/users/a48ce380d0d1451c82f6262f84461246/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:44.287448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.334932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042553a9-b95f-4ec5-88e9-664827aeb432 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.408228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.411999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:44.573761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:44.574461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.574740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.575029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.575150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7a4c7-3156-4e17-83ca-4685af33b5f2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.576033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:44.590022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:44.590022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.590270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.590740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] PATH_INFO: `/v3/users/6cdfa792e30e4527bf6f2006e00ab2fa` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.592839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f48473-fb3d-4bfa-ad06-057a3ed09058 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:44.593855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdfa792e30e4527bf6f2006e00ab2fa)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:44.596210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:44.640779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 915/1832] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:44.642432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.645675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.647638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.659493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 916/1833] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:08:44 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:44.660684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.666420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:44.718448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdfa792e30e4527bf6f2006e00ab2fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.720451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63444f-cb1e-4aa1-bddf-a04199a77727 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.732416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:44.733400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:44.733821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:44.734286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] PATH_INFO: `/v3/projects/c4a3862408814673a6a5c6b52ec2c55f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:44.740827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a3862408814673a6a5c6b52ec2c55f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:44.743228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:44.762846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdfa792e30e4527bf6f2006e00ab2fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.763611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdfa792e30e4527bf6f2006e00ab2fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.785107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdfa792e30e4527bf6f2006e00ab2fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.794612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdfa792e30e4527bf6f2006e00ab2fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.805465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 918/1834] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:44 2022] DELETE /identity/v3/users/6cdfa792e30e4527bf6f2006e00ab2fa => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:44.805465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:44.806397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:44.840105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:44.900798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a3862408814673a6a5c6b52ec2c55f', 'request_id': 'req-69b7b767-f8c2-498b-b579-5aba63df1b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.917834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a3862408814673a6a5c6b52ec2c55f', 'request_id': 'req-69b7b767-f8c2-498b-b579-5aba63df1b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.930936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a3862408814673a6a5c6b52ec2c55f', 'request_id': 'req-69b7b767-f8c2-498b-b579-5aba63df1b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.976574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1d40b-0eda-46d0-ba07-d3092b6281c4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:44.990301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] The token cache is being invalidate because project c4a3862408814673a6a5c6b52ec2c55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:44.990846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:44.997488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-69b7b767-f8c2-498b-b579-5aba63df1b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.000428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 917/1835] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:44 2022] DELETE /identity/v3/projects/c4a3862408814673a6a5c6b52ec2c55f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:45.003733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.004071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.005670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.065392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:45.067861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.148470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:45.164079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:45.166563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.167274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.167623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] PATH_INFO: `/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526/users/a48ce380d0d1451c82f6262f84461246/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.170358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a80fbc29cb4418a2c2b9975b9ff526, user_id=a48ce380d0d1451c82f6262f84461246, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:45.173238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fff32-e62d-4063-bb12-48c5f3e4ec26 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:45.187723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 919/1836] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:44 2022] PUT /identity/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526/users/a48ce380d0d1451c82f6262f84461246/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:45.189296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8542f8-b431-4668-92fe-5b4ccbad3436 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.189658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8542f8-b431-4668-92fe-5b4ccbad3436 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.190005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8542f8-b431-4668-92fe-5b4ccbad3436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.204658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 918/1837] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:45.209425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:45.210651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.225848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde644-0823-4c15-941b-6b04bcc6380f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:45.227855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-df8542f8-b431-4668-92fe-5b4ccbad3436 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:45.229736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df8542f8-b431-4668-92fe-5b4ccbad3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.379210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:45.379828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.384249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.385252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] PATH_INFO: `/v3/users/ccaf6f5b45c2438a94c1a98f6a0bd177` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.389820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccaf6f5b45c2438a94c1a98f6a0bd177)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:45.396034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:45.452449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 920/1838] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:45.458778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.459155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.459508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.516939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccaf6f5b45c2438a94c1a98f6a0bd177'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.528892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccaf6f5b45c2438a94c1a98f6a0bd177'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.551911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] MFA Rules not processed for user `a48ce380d0d1451c82f6262f84461246`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:45.552129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccaf6f5b45c2438a94c1a98f6a0bd177'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.555005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.563341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccaf6f5b45c2438a94c1a98f6a0bd177'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.564469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccaf6f5b45c2438a94c1a98f6a0bd177'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:45.573714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 919/1839] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:45 2022] DELETE /identity/v3/users/ccaf6f5b45c2438a94c1a98f6a0bd177 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:45.574840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:45.579909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.621896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:45.719075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1d18c-37a0-43f2-9a06-4110eb459f60 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:45.727315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:45.727976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.728258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.728546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.733989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 921/1840] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:45.733989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:45.733989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.735716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.758630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c7664-bd58-4541-bab8-af8c0bad1536 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:45.779778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:45.884311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:45.884572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.886657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.887299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.891802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.914273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:45.915617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdcfd9-a2db-4a9e-b3a2-48507570d8ce service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:45.921880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:45.924111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d131bbd-af0c-482f-90a3-d3d47ad710c1 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:45.936859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 920/1841] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:45.938425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.938788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.939188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:45.941407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 921/1842] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:45.942350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:45.945001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:45.953051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 922/1843] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:45.954545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:45.954837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:45.957211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.007222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:46.009921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.053862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 923/1844] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:46.080252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e8f96-5829-4dbf-8f48-cad41b9de4ff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.085403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.086915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.088307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.088726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.194222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.196030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.232161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e88676-ec16-4031-87f3-6dad2dd8f0fe None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.237946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.242563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.242868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.243205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] PATH_INFO: `/v3/projects/e26c8e1817784ea494c2ae0511cae3e9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.244879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26c8e1817784ea494c2ae0511cae3e9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.246956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.271924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.275229 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.293130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 922/1845] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:46.315772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.317522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.353194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b4e2a-7467-40a2-aacf-b88db17db294 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.355324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26c8e1817784ea494c2ae0511cae3e9', 'request_id': 'req-771bd5e3-8ae5-4550-9917-648eef46973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.377029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26c8e1817784ea494c2ae0511cae3e9', 'request_id': 'req-771bd5e3-8ae5-4550-9917-648eef46973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.390855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26c8e1817784ea494c2ae0511cae3e9', 'request_id': 'req-771bd5e3-8ae5-4550-9917-648eef46973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.419432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] The token cache is being invalidate because project e26c8e1817784ea494c2ae0511cae3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:46.423146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.424468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-771bd5e3-8ae5-4550-9917-648eef46973e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.428710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 924/1846] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:46 2022] DELETE /identity/v3/projects/e26c8e1817784ea494c2ae0511cae3e9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:46.433360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7244-8bb2-4420-8833-1d0a140de82e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.433861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7244-8bb2-4420-8833-1d0a140de82e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.434315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc7244-8bb2-4420-8833-1d0a140de82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.442974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:46.474148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.474786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.475068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.475345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.479068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.480163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c7dc7244-8bb2-4420-8833-1d0a140de82e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:46.481394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c7dc7244-8bb2-4420-8833-1d0a140de82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.503107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.505569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.519305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:46.535925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 923/1847] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:46.537006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.541401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.639089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 925/1848] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:46.646051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.646051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.646051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.663686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71388d-94eb-4d70-823b-c799f4eb5f67 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.679110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:46.679563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.680706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.681352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.682078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] PATH_INFO: `/v3/projects/63f9538c08c44912974abc080c389236` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.683942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] RBAC: Authorizing `identity:delete_project(project_id=63f9538c08c44912974abc080c389236)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.686762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.686893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.702269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 926/1849] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:46.710012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.728683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87077791-8170-40e2-8687-adf7ccf855e6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.735836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.737625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.739281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.739775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.741056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:46.742369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.745082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.773322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 927/1850] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:46.777595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.793377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5550-b727-41ee-990b-522740107987 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.795068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63f9538c08c44912974abc080c389236', 'request_id': 'req-ebd572fe-eaaf-4821-9f23-4a7936cf740d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.803185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63f9538c08c44912974abc080c389236', 'request_id': 'req-ebd572fe-eaaf-4821-9f23-4a7936cf740d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.811775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63f9538c08c44912974abc080c389236', 'request_id': 'req-ebd572fe-eaaf-4821-9f23-4a7936cf740d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.844862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] The token cache is being invalidate because project 63f9538c08c44912974abc080c389236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:46.845266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.846040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:46.851662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 924/1851] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:46 2022] DELETE /identity/v3/projects/63f9538c08c44912974abc080c389236 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:46.853942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.855456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.887614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:46.888326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:46.888745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:46.889064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:46.889734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:46.891929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:46.929003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 928/1852] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:46 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:46.933024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:46.934909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:46.938671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd572fe-eaaf-4821-9f23-4a7936cf740d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:46.961588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:47.015787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1548e5-2b40-457d-aea6-15d67593c3b0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.017992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.018834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.019122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.019914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] PATH_INFO: `/v3/users/f3676361044e47c589bf2906f58aac8e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.021971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3676361044e47c589bf2906f58aac8e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.023966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.025421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.026090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.026181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.026602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.028484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.030480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.055421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 929/1853] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:46 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:47.060573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.071574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce4ff1-44bf-48be-b11d-1a53d1380ee0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.077061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.077389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.077627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.077997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.078618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.080223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.092230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 930/1854] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:47.095983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.097009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3676361044e47c589bf2906f58aac8e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.108805 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3676361044e47c589bf2906f58aac8e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.110218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709095b-37b1-499a-b37a-dffac1be308e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.115746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.116527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.116818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.117140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.118476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.120857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.123552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:47.124594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3676361044e47c589bf2906f58aac8e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.132096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3676361044e47c589bf2906f58aac8e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.132826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3676361044e47c589bf2906f58aac8e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.140569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 931/1856] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:47.141042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 925/1856] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:46 2022] DELETE /identity/v3/users/f3676361044e47c589bf2906f58aac8e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:47.141473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.143854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.158323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd9775-cf1f-4611-924c-4dd77bb83533 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.185687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.187260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.233779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.234409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.234711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.235091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] PATH_INFO: `/v3/projects/7b2fb0584531459f8fde9efcb7364981/users/04d3c37b396c4c469ec5419e63878c82/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.253567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2fb0584531459f8fde9efcb7364981, user_id=04d3c37b396c4c469ec5419e63878c82, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.257249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244433c0-720d-4dee-a602-7f0766c786f9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.268350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9392-872a-4374-b944-77c75d1962f0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.276200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 932/1857] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:47 2022] PUT /identity/v3/projects/7b2fb0584531459f8fde9efcb7364981/users/04d3c37b396c4c469ec5419e63878c82/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:47.286689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.287120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.287407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.321087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] MFA Rules not processed for user `04d3c37b396c4c469ec5419e63878c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:47.322937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.334116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:47.355383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.356304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.357273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.357831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] PATH_INFO: `/v3/users/e0a117fc71364c7894d0b5497ce04c89` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.359101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a117fc71364c7894d0b5497ce04c89)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.361797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.404071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 933/1858] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:47.408863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.420841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a117fc71364c7894d0b5497ce04c89'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.422449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101a7d6-a483-49ec-ae3c-87e9390cd066 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.427880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.428607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.428984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.429297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.430053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.431675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a117fc71364c7894d0b5497ce04c89'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.431884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.448215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a117fc71364c7894d0b5497ce04c89'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.455568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a117fc71364c7894d0b5497ce04c89'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.456770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-603993fe-3654-4333-a5a1-1e1d3acb1a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a117fc71364c7894d0b5497ce04c89'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.465786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 926/1859] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:47 2022] DELETE /identity/v3/users/e0a117fc71364c7894d0b5497ce04c89 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:47.470594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.470935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.471496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.487239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 934/1860] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:47.490600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.492493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.495483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:47.497002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.568267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 927/1861] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:47.577415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:47.580291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.582935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.598369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5be3-435f-44b7-93cd-75c44f6a33e5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.603690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.604519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.605007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.605706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.606871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.609151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.656360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 935/1862] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:47.657371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.659740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.668025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d579ec-e7aa-4ac7-8de3-4e790a404116 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.673187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.673450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5611-ce73-4c6b-a916-e4dcc30d5899 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.674523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.674523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.674523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.680400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.681195 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.681555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.681909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] PATH_INFO: `/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.683185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b3f8cdc1b04519b78f32f8184fdf98)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.685124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.687204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.688810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.699886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 928/1863] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:47.701931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.726738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316275-9fa2-4e3d-8bb5-827d9a449679 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.739059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:47.755508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b3f8cdc1b04519b78f32f8184fdf98', 'request_id': 'req-4835a1b8-cf17-4d20-aefb-629eb581ca14'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.779874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b3f8cdc1b04519b78f32f8184fdf98', 'request_id': 'req-4835a1b8-cf17-4d20-aefb-629eb581ca14'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.795896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b3f8cdc1b04519b78f32f8184fdf98', 'request_id': 'req-4835a1b8-cf17-4d20-aefb-629eb581ca14'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.812170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] The token cache is being invalidate because project 74b3f8cdc1b04519b78f32f8184fdf98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:47.813120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.813729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4835a1b8-cf17-4d20-aefb-629eb581ca14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:47.815495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 936/1864] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:47 2022] DELETE /identity/v3/projects/74b3f8cdc1b04519b78f32f8184fdf98 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:47.819019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.820085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.820698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.821077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebebb8-afc1-409f-ba86-fc7cabdb6a56 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.821284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.821372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebebb8-afc1-409f-ba86-fc7cabdb6a56 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.821639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebebb8-afc1-409f-ba86-fc7cabdb6a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.823305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.825848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.840669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 929/1865] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:47.842391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.844408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.846243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-06ebebb8-afc1-409f-ba86-fc7cabdb6a56 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:47.848340 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06ebebb8-afc1-409f-ba86-fc7cabdb6a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.860150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e894d8-e6bb-4f5c-b268-dcae68fade22 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.871516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 937/1866] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:47.873238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:47.874333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.874333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.874333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.874752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:47.875107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:47.875704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] PATH_INFO: `/v3/projects/d3acfcd0ea044baeb2402e02c336d29e/users/28a193951a8949ef97cdb26309477cfe/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:47.897073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:47.898614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:47.906455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3acfcd0ea044baeb2402e02c336d29e, user_id=28a193951a8949ef97cdb26309477cfe, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:47.911128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:47.912485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 938/1867] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:47.913436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:47.927244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3af5bd-8380-4aa3-94fa-b4c0146bd343 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:47.932311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 930/1868] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:47 2022] PUT /identity/v3/projects/d3acfcd0ea044baeb2402e02c336d29e/users/28a193951a8949ef97cdb26309477cfe/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:47.936864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.025515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.026178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.026489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.026769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.027623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:48.027937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.029667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.032761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef81c1-a755-4d58-83df-322f3767cf76 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.040927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.041739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.042122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.042466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.043238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.045122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.046362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 939/1869] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:48.047612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.058486 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c56e1-8219-4f91-a4d6-d6b8669c8767 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.059481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 931/1870] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:48.063395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.067949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.068826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.069159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.069442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.070123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.072055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.089486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458d434-0b4c-41ba-a9ee-9b250c68c29a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.089486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.092763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.093077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.093354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] PATH_INFO: `/v3/projects/d3acfcd0ea044baeb2402e02c336d29e/users/28a193951a8949ef97cdb26309477cfe/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.095566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3acfcd0ea044baeb2402e02c336d29e, user_id=28a193951a8949ef97cdb26309477cfe, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.098941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40069cf-2b88-4b85-b3e4-2edad246fd20 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.105532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 940/1871] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:48 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:48.112003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.117003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 932/1872] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:48 2022] PUT /identity/v3/projects/d3acfcd0ea044baeb2402e02c336d29e/users/28a193951a8949ef97cdb26309477cfe/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:48.127922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064aa51-031e-450b-8b43-815445f48150 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.128220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064aa51-031e-450b-8b43-815445f48150 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.128613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064aa51-031e-450b-8b43-815445f48150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.162657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:48.168781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a064aa51-031e-450b-8b43-815445f48150 None None] MFA Rules not processed for user `28a193951a8949ef97cdb26309477cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:48.170441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a064aa51-031e-450b-8b43-815445f48150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.210901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e487d6-ebb1-4d51-a831-8500c3e8939e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.215577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.216192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.216472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.216768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.217477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.219392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.226294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:48.244743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 941/1873] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:48 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:48.244743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.244743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.250706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 933/1874] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:48.251542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064aa51-031e-450b-8b43-815445f48150 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.257790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5644d24-862b-48db-a3e8-182e0cbfd30f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.262610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.263305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.263507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.263787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] PATH_INFO: `/v3/projects/ef528c41502b4b13bb9f88a474503a01` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.265003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef528c41502b4b13bb9f88a474503a01)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.265887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064aa51-031e-450b-8b43-815445f48150 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.266928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.271815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.272431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.272772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.273094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.273786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.275922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.286047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 934/1875] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:48.286879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.291737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.323510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef528c41502b4b13bb9f88a474503a01', 'request_id': 'req-750d9d4d-a266-4634-b227-d81da750298f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.333532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef528c41502b4b13bb9f88a474503a01', 'request_id': 'req-750d9d4d-a266-4634-b227-d81da750298f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.341124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef528c41502b4b13bb9f88a474503a01', 'request_id': 'req-750d9d4d-a266-4634-b227-d81da750298f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.356408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] The token cache is being invalidate because project ef528c41502b4b13bb9f88a474503a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:48.356952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.357839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.359600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 942/1876] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:48 2022] DELETE /identity/v3/projects/ef528c41502b4b13bb9f88a474503a01 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:48.360366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.361926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.368506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d713c0a-191e-4483-9987-63f97504b4c5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.446153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d9d4d-a266-4634-b227-d81da750298f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.451223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.451841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.452119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.452505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.453226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.454220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.454853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.454961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.455282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.455625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.457558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.468642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 943/1877] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:48.468642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.470039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.493481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406944d-4cba-4198-a0fa-ec1d22a72482 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.514152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.514730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.515150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.515432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] PATH_INFO: `/v3/users/a48ce380d0d1451c82f6262f84461246` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.517730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48ce380d0d1451c82f6262f84461246)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.519843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.545099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:48.557630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.559095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.585540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 935/1878] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:48.586668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:48.621160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836533-dfd7-48bb-a98f-428bcf153dd3 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:48.624508 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48ce380d0d1451c82f6262f84461246'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.639373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:48.640088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.640375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.640658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] PATH_INFO: `/v3/projects/7ac36474c15d403d92572c970bd6686d/users/f25d68d9695141cca8485232a9a9a332/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.655149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48ce380d0d1451c82f6262f84461246'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.657650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48ce380d0d1451c82f6262f84461246'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.670092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48ce380d0d1451c82f6262f84461246'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.673857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac36474c15d403d92572c970bd6686d, user_id=f25d68d9695141cca8485232a9a9a332, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:48.680239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766467e-fb92-4ebc-8df3-4e1dd642933f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:48.697933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48ce380d0d1451c82f6262f84461246'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:48.714660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 936/1879] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:48 2022] PUT /identity/v3/projects/7ac36474c15d403d92572c970bd6686d/users/f25d68d9695141cca8485232a9a9a332/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:48.718089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 944/1880] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:48 2022] DELETE /identity/v3/users/a48ce380d0d1451c82f6262f84461246 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:48.733211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:48.733568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:48.733960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:48.825008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] MFA Rules not processed for user `f25d68d9695141cca8485232a9a9a332`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:48.827046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:48.856161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:49.013414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 937/1881] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:49.023365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.025121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:49.219371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16640ac9-270f-447c-8867-8e0a934975ba admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:49.246245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:49.247368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.253711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.255268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.256333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:49.258795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:49.356058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 945/1882] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:49 2022] POST /identity/v3/projects => generated 381 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:49.363556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.370988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.373001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:49.385298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76a15a-f4da-4f90-9ebc-b6dca3ec4aef None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:49.510698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c995083-3b7a-4de5-8428-26900aa9ae73 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:49.515430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:49.516075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.516362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.516640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.517352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:49.519418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:49.532609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:49.533214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.533497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.533953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] PATH_INFO: `/v3/users/04d3c37b396c4c469ec5419e63878c82` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.535307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d3c37b396c4c469ec5419e63878c82)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:49.537687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:49.585335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 938/1883] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:49 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:49.588013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.614611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc26b6-c812-4cda-b341-fda99d3f56ce admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:49.621185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:49.624248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:49.624911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.625413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.625832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.627175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:49.629428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:49.658350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 939/1884] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:49.662630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.682149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d3c37b396c4c469ec5419e63878c82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:49.685099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea7b81-06c7-404f-8a54-910af8a96a59 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:49.698542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:49.700469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.700929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.701312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] PATH_INFO: `/v3/projects/58a0067688e24da799b9428c13bfaad8/users/7a8c23ad170a4e5f9dda98ab37413f76/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.739068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d3c37b396c4c469ec5419e63878c82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:49.739937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a0067688e24da799b9428c13bfaad8, user_id=7a8c23ad170a4e5f9dda98ab37413f76, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:49.740190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d3c37b396c4c469ec5419e63878c82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:49.750731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:49.754657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d3c37b396c4c469ec5419e63878c82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:49.784389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36eb26ed-7c4a-4a6e-90c9-62b7c35644d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d3c37b396c4c469ec5419e63878c82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:49.815838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 940/1885] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:49 2022] PUT /identity/v3/projects/58a0067688e24da799b9428c13bfaad8/users/7a8c23ad170a4e5f9dda98ab37413f76/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:49.817043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 946/1886] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:49 2022] DELETE /identity/v3/users/04d3c37b396c4c469ec5419e63878c82 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:49.820443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:49.824062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:49.824926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:49.826128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:49.828655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:49.876057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:49.877948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:49.985276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 947/1887] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:49.998822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0690c-2bc5-46e1-b788-d79825e39094 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.016581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.017554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.017903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.018184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.018916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:50.021425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f243ec4-a9e8-44be-be44-721d834b066c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:50.032811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:50.056646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 941/1888] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:50.057447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:50.076039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b573-a886-42db-b1c8-7fd405a5f01d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.095124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.095897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.096433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.096767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] PATH_INFO: `/v3/projects/58a0067688e24da799b9428c13bfaad8/users/7a8c23ad170a4e5f9dda98ab37413f76/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.099314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a0067688e24da799b9428c13bfaad8, user_id=7a8c23ad170a4e5f9dda98ab37413f76, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:50.107804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:50.142296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 948/1889] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:50 2022] PUT /identity/v3/projects/58a0067688e24da799b9428c13bfaad8/users/7a8c23ad170a4e5f9dda98ab37413f76/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:50.153869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.154214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.154519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.159471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:50.162198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:50.224504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] MFA Rules not processed for user `7a8c23ad170a4e5f9dda98ab37413f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:50.229050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:50.417070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0052ee-4ca2-457f-b66c-22cb78e6659b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.434008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.437007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.437303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.437582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.469149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 942/1890] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:50.475573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:50.477232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:50.622514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba21593-7409-4a8e-91fd-29d358db3d9c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.629524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:50.644929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.645779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.646129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.646415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.669590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:50.675613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:50.682048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:50.687172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:50.703664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 949/1891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:50.743841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 943/1892] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:50.892309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:50.893935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:50.894068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:50.906337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183266-a57f-412f-bdb5-42a2a287593e service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.912484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.913611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.913910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.914180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.916224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:50.917262 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d27962-8855-49e4-a2c3-9eabb8e7e8dc service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:50.946305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:50.947705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:50.947992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:50.948276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] PATH_INFO: `/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:50.953282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a80fbc29cb4418a2c2b9975b9ff526)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:50.955561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.019465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.021272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf14850-42e8-43ec-9a74-57690e9bd2dd service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.031832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 944/1893] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:51.035640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a80fbc29cb4418a2c2b9975b9ff526', 'request_id': 'req-07ddfad6-0311-47f3-bb80-c50c17360ac4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.044052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a80fbc29cb4418a2c2b9975b9ff526', 'request_id': 'req-07ddfad6-0311-47f3-bb80-c50c17360ac4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.053804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a80fbc29cb4418a2c2b9975b9ff526', 'request_id': 'req-07ddfad6-0311-47f3-bb80-c50c17360ac4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.069905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] The token cache is being invalidate because project 15a80fbc29cb4418a2c2b9975b9ff526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:51.070337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.071113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07ddfad6-0311-47f3-bb80-c50c17360ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.073380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 950/1894] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:50 2022] DELETE /identity/v3/projects/15a80fbc29cb4418a2c2b9975b9ff526 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:51.078874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.079389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.079870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.112358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:51.115149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:51.152714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:51.204057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 945/1895] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:51.211587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.211878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.212159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.244679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:51.245738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:51.260622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 951/1896] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:51.264596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.276693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87823e-80cb-4b08-8777-f5c434d3e44d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.284014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.287560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.287837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.288319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.289292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:51.289606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.291984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.311249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 946/1897] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:51.312438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.329127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72deca1-6bf5-42e3-85de-4aafa3bc9ac1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.337549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.338376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.339034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.339496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.340337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.343371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.374616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 952/1898] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:51.378797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.390291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad943e4-f71a-434d-b6b4-2aa7339ce671 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.470658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.471305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.471583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.471952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.474001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.477252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.503094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 947/1899] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:51.507973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.525609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abd160-8d36-40fd-83a9-c7115ed3c1c4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.537964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.539465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.539843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.540200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.541322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.543427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.560311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 953/1900] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:51.562234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.573880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd84d21-45b4-454a-8f72-995a9f7b1b1a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.579519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.580244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.580947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.581412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.583418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.585205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061f16-97fa-4076-ab57-67f1cf832dc0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.605627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 948/1901] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:51.622244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.637559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa2424-3cc3-4b35-bceb-ffe1cf38a8b2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.643084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.643946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.644072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.644920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] PATH_INFO: `/v3/projects/5a84deae35c74b48b53542076823ba60/users/59d18a7a2dbd47e799b901b208084a68/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.662308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a84deae35c74b48b53542076823ba60, user_id=59d18a7a2dbd47e799b901b208084a68, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.664954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.682628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 954/1902] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:51 2022] PUT /identity/v3/projects/5a84deae35c74b48b53542076823ba60/users/59d18a7a2dbd47e799b901b208084a68/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:51.694040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.695317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.695790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.736271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] MFA Rules not processed for user `59d18a7a2dbd47e799b901b208084a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:51.738694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:51.767679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.770347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:51.849159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 949/1903] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:51.850163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:51.883077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece32e7-ee92-45bf-a4ab-6f3499e2cd17 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.887683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:51.889593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:51.890333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:51.890623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:51.890905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] PATH_INFO: `/v3/projects/7b2fb0584531459f8fde9efcb7364981` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:51.893345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2fb0584531459f8fde9efcb7364981)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:51.896739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:51.899626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:51.967314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999bdb4-2745-4eda-b5a5-9cccb137cd12 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:51.984913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2fb0584531459f8fde9efcb7364981', 'request_id': 'req-30b01980-fc8d-4de3-a85f-7b036bce81f6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:51.993528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2fb0584531459f8fde9efcb7364981', 'request_id': 'req-30b01980-fc8d-4de3-a85f-7b036bce81f6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.001474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2fb0584531459f8fde9efcb7364981', 'request_id': 'req-30b01980-fc8d-4de3-a85f-7b036bce81f6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.016232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] The token cache is being invalidate because project 7b2fb0584531459f8fde9efcb7364981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:52.016530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.017286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.019451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 955/1904] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:51 2022] DELETE /identity/v3/projects/7b2fb0584531459f8fde9efcb7364981 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:52.023704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.025155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:52.044534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.046084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.047169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.047663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.048516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.051539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.090659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 950/1905] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:51 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:52.091734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.093823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:52.094532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b01980-fc8d-4de3-a85f-7b036bce81f6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.182683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc203b-a7de-4ac8-b1cc-0f76cf7f02ee admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.188005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.188837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.189253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.189641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.190580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.192862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.197090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.198097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.199065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.199665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] PATH_INFO: `/v3/users/28a193951a8949ef97cdb26309477cfe` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.201575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a193951a8949ef97cdb26309477cfe)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.207310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.223667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 951/1906] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:52 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:52.227900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.253691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b8a70-f4d8-471f-817c-5810fcebe404 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.263772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.264396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.264675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.264946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.265664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.267313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.275118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:52.292585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a193951a8949ef97cdb26309477cfe'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.297414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 952/1907] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:52.299257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.311059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246baa0-efcd-4c09-ab15-8c550d695139 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.317185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.317914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.318450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.318726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] PATH_INFO: `/v3/projects/2e04338840c940f8b7a5542e3bba8332/users/93f76e4b02a7452088e1553857879c99/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.322312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a193951a8949ef97cdb26309477cfe'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.323418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a193951a8949ef97cdb26309477cfe'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.334308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a193951a8949ef97cdb26309477cfe'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.338515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e04338840c940f8b7a5542e3bba8332, user_id=93f76e4b02a7452088e1553857879c99, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.342554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.344762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a193951a8949ef97cdb26309477cfe'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:52.361600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 956/1908] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:52 2022] DELETE /identity/v3/users/28a193951a8949ef97cdb26309477cfe => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:52.369345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 953/1909] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:52 2022] PUT /identity/v3/projects/2e04338840c940f8b7a5542e3bba8332/users/93f76e4b02a7452088e1553857879c99/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:52.373474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.374908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:52.452900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706e5e8-cfde-44a9-a7ef-03be14eb7272 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.458274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.458999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.459603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.459603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.460429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.462130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4877a0-19ff-4236-bbc1-ac981dac1483 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.477164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 957/1910] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:52.481211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.495544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:52.565910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678630a9-e132-4340-a9da-4b64d4addd31 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.571742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.572408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.572685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.572955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] PATH_INFO: `/v3/projects/2e04338840c940f8b7a5542e3bba8332/users/93f76e4b02a7452088e1553857879c99/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.574912 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e04338840c940f8b7a5542e3bba8332, user_id=93f76e4b02a7452088e1553857879c99, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.577629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.589361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 954/1911] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:52 2022] PUT /identity/v3/projects/2e04338840c940f8b7a5542e3bba8332/users/93f76e4b02a7452088e1553857879c99/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:52.601914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.602197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.602496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.636515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] MFA Rules not processed for user `93f76e4b02a7452088e1553857879c99`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:52.637867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:52.760159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 958/1912] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:52.774276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.777347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:52.854620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06a2e7-6b1a-4067-a46e-b7accc5eb065 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.858478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.859117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.859395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.859668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.873979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:52.875444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:52.887406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 955/1913] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:52.967432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:52.979976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418c9a-0a2b-4429-a9b0-4dbe69bce240 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:52.985591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:52.986343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:52.986692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:52.987029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:52.989550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.037109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:53.075826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.077892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.102395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 959/1914] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:53.308256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.310444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.395360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199e38f-77e4-420f-ab74-79a2106d8ecc service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.407039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.407713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.408133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.408472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] PATH_INFO: `/v3/projects/d3acfcd0ea044baeb2402e02c336d29e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.409710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3acfcd0ea044baeb2402e02c336d29e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.413396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.450231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.463332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225520-76d6-4cec-add8-c618a7dd87c5 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.477659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:53.478467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3acfcd0ea044baeb2402e02c336d29e', 'request_id': 'req-119549a0-8afc-4dd2-ba2b-8dec55711787'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.485987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3acfcd0ea044baeb2402e02c336d29e', 'request_id': 'req-119549a0-8afc-4dd2-ba2b-8dec55711787'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.493464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3acfcd0ea044baeb2402e02c336d29e', 'request_id': 'req-119549a0-8afc-4dd2-ba2b-8dec55711787'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.506832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] The token cache is being invalidate because project d3acfcd0ea044baeb2402e02c336d29e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:53.507146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.507834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-119549a0-8afc-4dd2-ba2b-8dec55711787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.509503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 956/1915] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:53 2022] DELETE /identity/v3/projects/d3acfcd0ea044baeb2402e02c336d29e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:53.514607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89d3d-b8dc-4329-bccd-6417dcc2744e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.514898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89d3d-b8dc-4329-bccd-6417dcc2744e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.515178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89d3d-b8dc-4329-bccd-6417dcc2744e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.536176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-25d89d3d-b8dc-4329-bccd-6417dcc2744e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:53.537445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-25d89d3d-b8dc-4329-bccd-6417dcc2744e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.574207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.581320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.582940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.583141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] PATH_INFO: `/v3/users/59d18a7a2dbd47e799b901b208084a68` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.586094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d18a7a2dbd47e799b901b208084a68)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.590135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.624129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 957/1916] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:53.633667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.634233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.634233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.669747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d18a7a2dbd47e799b901b208084a68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.672164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:53.673414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.682045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 958/1917] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:53.683572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d18a7a2dbd47e799b901b208084a68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.684319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d18a7a2dbd47e799b901b208084a68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.691305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d18a7a2dbd47e799b901b208084a68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.696061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.699770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-02268c15-dd27-4cdb-a1be-48e066b29d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d18a7a2dbd47e799b901b208084a68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:53.707256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16627b12-89ef-459d-b3ad-2c630f33717b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.707440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 960/1918] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:53 2022] DELETE /identity/v3/users/59d18a7a2dbd47e799b901b208084a68 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:53.711773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.712063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.712339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.738898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:53.740192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.777847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.778479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.778768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.779051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.780561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:53.780715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.782566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.797282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 959/1919] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:08:53.800615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.802891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.805330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:53.817543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ae1ce-e9c5-4f61-b97a-5437ac6ab245 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.820496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 961/1920] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:53.823335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.824100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.824417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.824612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.825294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.827252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.830440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.832541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:53.859410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 960/1921] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:53 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:53.864440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.905081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699bd20-510a-4ce4-a69e-c2e74271ef2a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.916539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:53.937702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaa012-a5db-4132-8a07-efa52819fa13 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.943587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.944150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.944584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.944869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.945576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.947518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.967813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.968446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.968724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.969020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.972906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 961/1922] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:53 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:53.976698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:53.982527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.983973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:53.986748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421554f-435d-4861-8816-82e4e47a84d3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:53.993247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:53.994350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:53.994470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 962/1923] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:53.997317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:53.997434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:53.998089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:53.999714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.011979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 962/1924] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:54.016633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.028228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324d49-0e63-4095-98a9-b95eec43db8b service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.033751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.034448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.034845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.035259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.036137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.037907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02f565-983b-4468-b121-5cec815fc43d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.052193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 963/1925] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:54.053815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.065933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467d87b-619d-4559-b6b9-7fd6a13eed91 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.076547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.077813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.078628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.078628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] PATH_INFO: `/v3/projects/e0d03f6f3f8b4086b8f52846edc4d5ad/users/8d91476914e34fc8a39f750c3cdff93f/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.103211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d03f6f3f8b4086b8f52846edc4d5ad, user_id=8d91476914e34fc8a39f750c3cdff93f, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.106580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.119846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 963/1926] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:54 2022] PUT /identity/v3/projects/e0d03f6f3f8b4086b8f52846edc4d5ad/users/8d91476914e34fc8a39f750c3cdff93f/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:54.131438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.131730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.132017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.163326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] MFA Rules not processed for user `8d91476914e34fc8a39f750c3cdff93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:54.164926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.241343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 964/1927] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:54.243099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.337885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861426e0-d2a1-41d3-bb15-971740dedf23 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.342930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.343556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.343850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.344128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.344801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.346786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.399619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 964/1928] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:54 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:54.403740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.417423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89965c61-2906-4ebb-abc7-2dfd4f80d213 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.452832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:54.461154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.462937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.491003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.491642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.491920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.492192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.492862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.495512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.521771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 965/1929] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:54 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:54.525333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.536758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a256e-626f-4c49-8411-62b18b85194f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.541792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.542488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.542779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.543059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.543756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.545519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.550116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d367220-5837-4956-bf50-480da0d4dfc9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.556054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.556777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.560139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.560255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 966/1930] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:54.560367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] PATH_INFO: `/v3/projects/5a84deae35c74b48b53542076823ba60` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.560367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a84deae35c74b48b53542076823ba60)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.560600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.561191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.576216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed2d1-eb5e-4e3a-abb1-1181028eaeee admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.581363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.582767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.583636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.583636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] PATH_INFO: `/v3/projects/8369109497144a85a3b26a2c6be85cbc/users/882ca5c6540240a988c08197c1715500/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.593231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:54.599727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8369109497144a85a3b26a2c6be85cbc, user_id=882ca5c6540240a988c08197c1715500, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.602642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.621703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 967/1931] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:54 2022] PUT /identity/v3/projects/8369109497144a85a3b26a2c6be85cbc/users/882ca5c6540240a988c08197c1715500/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:54.625715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.629957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a84deae35c74b48b53542076823ba60', 'request_id': 'req-3c375c84-a472-49cb-87b9-a47a1d120331'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.635453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a84deae35c74b48b53542076823ba60', 'request_id': 'req-3c375c84-a472-49cb-87b9-a47a1d120331'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.641561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a84deae35c74b48b53542076823ba60', 'request_id': 'req-3c375c84-a472-49cb-87b9-a47a1d120331'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.658145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] The token cache is being invalidate because project 5a84deae35c74b48b53542076823ba60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:54.658428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.659236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.661632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 965/1932] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:54 2022] DELETE /identity/v3/projects/5a84deae35c74b48b53542076823ba60 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:54.665402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.666807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.706550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da336959-84d1-4624-ac39-3ec8035ebd99 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.756143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c375c84-a472-49cb-87b9-a47a1d120331 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.764371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.765066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.765373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.765620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] PATH_INFO: `/v3/users/93f76e4b02a7452088e1553857879c99` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.767809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f76e4b02a7452088e1553857879c99)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.769974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.821405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.822569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.822860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.823139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.824309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.825877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.838044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 968/1933] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:54.840072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f76e4b02a7452088e1553857879c99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.842533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.845314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.850174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f76e4b02a7452088e1553857879c99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.859371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cd38-9e05-46d0-9774-890d40afd676 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:54.866303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f76e4b02a7452088e1553857879c99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.866621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:54.867212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.867768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.868057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] PATH_INFO: `/v3/projects/8369109497144a85a3b26a2c6be85cbc/users/882ca5c6540240a988c08197c1715500/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.870826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8369109497144a85a3b26a2c6be85cbc, user_id=882ca5c6540240a988c08197c1715500, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:54.873933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:54.876411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f76e4b02a7452088e1553857879c99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.877238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8a78a9c7-a745-4893-88da-46a5a7b4a4a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f76e4b02a7452088e1553857879c99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:54.889056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 966/1934] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:54 2022] DELETE /identity/v3/users/93f76e4b02a7452088e1553857879c99 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:54.901086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 969/1935] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:54 2022] PUT /identity/v3/projects/8369109497144a85a3b26a2c6be85cbc/users/882ca5c6540240a988c08197c1715500/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:54.906280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:54.906639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:54.907417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:54.920799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:54.922810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.955975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] MFA Rules not processed for user `882ca5c6540240a988c08197c1715500`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:54.958399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:54.961664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:54.984279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:55.031418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efba1f-475c-471b-87f3-f2d28db22914 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.080440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 967/1936] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:55.091688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:55.093917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.119861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.120619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.121588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.121772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] PATH_INFO: `/v3/users/f25d68d9695141cca8485232a9a9a332` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.123282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25d68d9695141cca8485232a9a9a332)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.125301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:55.179505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c80e2-d784-4143-894f-4290c21166fd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.184514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25d68d9695141cca8485232a9a9a332'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.188699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.190067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.190624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.191815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.194601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.200244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25d68d9695141cca8485232a9a9a332'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.201097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25d68d9695141cca8485232a9a9a332'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.211860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25d68d9695141cca8485232a9a9a332'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.223590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-840ab46b-60f2-4fed-abb8-f681450ea4a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25d68d9695141cca8485232a9a9a332'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.233277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 970/1937] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:54 2022] DELETE /identity/v3/users/f25d68d9695141cca8485232a9a9a332 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:55.242084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.243117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.243585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.265653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:55.267194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.286161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.287746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:55.306596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 968/1938] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:55.330763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:55.364470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 971/1939] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:55.392463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:55.395479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.415921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:55.417666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.454546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c75b7e-e9ff-4793-9c89-81050931d090 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.461454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.462278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.462571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.462906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] PATH_INFO: `/v3/projects/2e04338840c940f8b7a5542e3bba8332` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.465016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e04338840c940f8b7a5542e3bba8332)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.468463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:55.520161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5edf74-071d-42e6-bbfb-d56731ccecb9 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.530717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e04338840c940f8b7a5542e3bba8332', 'request_id': 'req-dd484d09-5e43-4283-a0e2-04521d1c3e71'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.537343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e04338840c940f8b7a5542e3bba8332', 'request_id': 'req-dd484d09-5e43-4283-a0e2-04521d1c3e71'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.543771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e04338840c940f8b7a5542e3bba8332', 'request_id': 'req-dd484d09-5e43-4283-a0e2-04521d1c3e71'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.576126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] The token cache is being invalidate because project 2e04338840c940f8b7a5542e3bba8332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:55.576802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.577914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd484d09-5e43-4283-a0e2-04521d1c3e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:55.579767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 972/1940] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:55 2022] DELETE /identity/v3/projects/2e04338840c940f8b7a5542e3bba8332 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:55.584906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:55.587023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88afa-ec2f-4900-b65b-42dbd79601b9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.587366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88afa-ec2f-4900-b65b-42dbd79601b9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.587696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee88afa-ec2f-4900-b65b-42dbd79601b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.627484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8ee88afa-ec2f-4900-b65b-42dbd79601b9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:55.630006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8ee88afa-ec2f-4900-b65b-42dbd79601b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.645722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.646369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.646653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.646933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.648825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.729723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 973/1941] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:55.731285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.731577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.732610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.799043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.801692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:55.814563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:55.815951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:55.834106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 969/1942] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:08:55.851861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 974/1943] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:55.856563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:55.873954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7200182-b312-41d5-9d48-bdfb18e1e67c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.893821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.893821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.893821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.893821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.894667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:55.894667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.899708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:55.930503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 970/1944] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:08:55.941706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:55.952625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6de82-5cc7-47a7-803c-b01d8258ab3a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:55.961180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:55.979026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:55.979468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:55.979830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:55.980640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:55.987079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.049006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 975/1945] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:55 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:56.053241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.064879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eaf1e-8b92-4f4d-9c2c-0e90ee51d6bb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.167195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.168017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.168374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.169065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.169869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.173145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.212648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.214554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 971/1946] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:56 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:56.227456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769453f0-2a0f-414e-ae1f-8778043eadff admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.234793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.235828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.236245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.236683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.237688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.240882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.278320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 976/1947] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:56.282268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.298794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2516de-6735-4d99-806a-898aab23ee2b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.310548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.311068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.311581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.311871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.313533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.315276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c9dd3-2326-46fc-9c0c-d2b1e9639a2b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.327609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 972/1948] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:56.332574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.352793 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246d1be-ba60-4efe-9139-c3a298d3d68c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.361998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.363540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.369121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.369730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] PATH_INFO: `/v3/projects/c2cebf9e504146238004adadde60ef1c/users/0ebeb4a1727f4472ae9a0a0890fced1b/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.392369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cebf9e504146238004adadde60ef1c, user_id=0ebeb4a1727f4472ae9a0a0890fced1b, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.398053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.420902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 977/1949] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:56 2022] PUT /identity/v3/projects/c2cebf9e504146238004adadde60ef1c/users/0ebeb4a1727f4472ae9a0a0890fced1b/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:56.433365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.433588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.433749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.467145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] MFA Rules not processed for user `0ebeb4a1727f4472ae9a0a0890fced1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:56.469335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:56.525183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:56.590283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 973/1950] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:56.600389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.602293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:56.615142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:56.682216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b99928-ad8b-4175-8920-b32b855161bf admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.689125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.689771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.690051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.690323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.711428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.713485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.723192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 978/1951] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:56.751937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.753312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.754058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:56.829824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3322ed3-0cc1-4638-a292-f199a019d92c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.831387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6406c4a-1b73-433c-aaab-76f746da8790 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:56.834829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.835511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.835759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.836040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] PATH_INFO: `/v3/projects/7ac36474c15d403d92572c970bd6686d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.836139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:56.836672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.836952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.837250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac36474c15d403d92572c970bd6686d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.837346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] PATH_INFO: `/v3/users/0ebeb4a1727f4472ae9a0a0890fced1b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.838438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ebeb4a1727f4472ae9a0a0890fced1b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:56.839160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.840348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:56.887831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac36474c15d403d92572c970bd6686d', 'request_id': 'req-9fca4250-e863-4d72-9c1c-417c8c375b5f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.890526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ebeb4a1727f4472ae9a0a0890fced1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.894754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac36474c15d403d92572c970bd6686d', 'request_id': 'req-9fca4250-e863-4d72-9c1c-417c8c375b5f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.902163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac36474c15d403d92572c970bd6686d', 'request_id': 'req-9fca4250-e863-4d72-9c1c-417c8c375b5f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.906274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ebeb4a1727f4472ae9a0a0890fced1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.907246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ebeb4a1727f4472ae9a0a0890fced1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.916205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ebeb4a1727f4472ae9a0a0890fced1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.920777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] The token cache is being invalidate because project 7ac36474c15d403d92572c970bd6686d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:56.921297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.922284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.923672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6883add-7e62-4f82-9b59-b3896caa2bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ebeb4a1727f4472ae9a0a0890fced1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:56.927674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 974/1952] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:56 2022] DELETE /identity/v3/projects/7ac36474c15d403d92572c970bd6686d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:56.928565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:56.929946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:56.931648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 979/1953] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:56 2022] DELETE /identity/v3/users/0ebeb4a1727f4472ae9a0a0890fced1b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:56.936096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:56.936451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:56.936796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:56.969894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:56.971239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:57.001129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:57.021248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4250-e863-4d72-9c1c-417c8c375b5f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:57.029518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:57.054135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 980/1954] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:57.077601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:57.086315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:57.140749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:57.141770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:57.142253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:57.142719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] PATH_INFO: `/v3/users/7a8c23ad170a4e5f9dda98ab37413f76` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:57.145267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8c23ad170a4e5f9dda98ab37413f76)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:57.148770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:57.185637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576fe46-fb06-498d-9959-ba4dad805f22 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:57.189868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:57.190504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:57.191248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:57.191564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:57.204244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:57.206112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:57.220292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 981/1955] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:57.246880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8c23ad170a4e5f9dda98ab37413f76'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:57.261803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8c23ad170a4e5f9dda98ab37413f76'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:57.278569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8c23ad170a4e5f9dda98ab37413f76'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:57.288545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8c23ad170a4e5f9dda98ab37413f76'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:57.289484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8c23ad170a4e5f9dda98ab37413f76'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:57.298573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 975/1956] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:08:56 2022] DELETE /identity/v3/users/7a8c23ad170a4e5f9dda98ab37413f76 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:57.830891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:57.838294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:57.840224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:57.852887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83845e5-481b-44fa-b3fe-b9ca13544a0c service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:57.927395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696313ed-ab23-4767-af5e-467c753f259b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:57.933524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:57.934286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:57.934477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:57.938377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] PATH_INFO: `/v3/projects/c2cebf9e504146238004adadde60ef1c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:57.938377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cebf9e504146238004adadde60ef1c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:57.939568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:57.982304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:57.983067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:57.983467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:57.983814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] PATH_INFO: `/v3/projects/58a0067688e24da799b9428c13bfaad8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:57.985120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a0067688e24da799b9428c13bfaad8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:57.987766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.027248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:58.047838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cebf9e504146238004adadde60ef1c', 'request_id': 'req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.054515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a0067688e24da799b9428c13bfaad8', 'request_id': 'req-60daf3c9-d06e-4520-8c0c-e391dd40b301'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.054962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cebf9e504146238004adadde60ef1c', 'request_id': 'req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.060087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a0067688e24da799b9428c13bfaad8', 'request_id': 'req-60daf3c9-d06e-4520-8c0c-e391dd40b301'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.061973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cebf9e504146238004adadde60ef1c', 'request_id': 'req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.066830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a0067688e24da799b9428c13bfaad8', 'request_id': 'req-60daf3c9-d06e-4520-8c0c-e391dd40b301'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.076307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] The token cache is being invalidate because project c2cebf9e504146238004adadde60ef1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:58.076652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.077254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f537ee55-f867-4b57-bcbe-0c36cfb4a54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.079013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 976/1957] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:57 2022] DELETE /identity/v3/projects/c2cebf9e504146238004adadde60ef1c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:58.083277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] The token cache is being invalidate because project 58a0067688e24da799b9428c13bfaad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:08:58.083605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.084335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-60daf3c9-d06e-4520-8c0c-e391dd40b301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:08:58.085241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bc38-3b15-45e1-9eef-ddb108a1eb2a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.085531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bc38-3b15-45e1-9eef-ddb108a1eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.085809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9bc38-3b15-45e1-9eef-ddb108a1eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.086323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 982/1958] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:08:57 2022] DELETE /identity/v3/projects/58a0067688e24da799b9428c13bfaad8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:58.091193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24b0c-7801-4270-b2c1-bcd1bb4d6c2d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.091660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24b0c-7801-4270-b2c1-bcd1bb4d6c2d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.092006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24b0c-7801-4270-b2c1-bcd1bb4d6c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.102068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:58.108306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a5c9bc38-3b15-45e1-9eef-ddb108a1eb2a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.109585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a5c9bc38-3b15-45e1-9eef-ddb108a1eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.115962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fef24b0c-7801-4270-b2c1-bcd1bb4d6c2d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.117574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fef24b0c-7801-4270-b2c1-bcd1bb4d6c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.170620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 977/1959] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:58.178482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.178776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.179090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.179615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 983/1960] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:58.188208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.188500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.188779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.200184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.201443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.209490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.210675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.210837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 978/1961] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:58.215931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.223039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 984/1962] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:58.226311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.229726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffc72c-c301-4cb3-b432-ba2619ae9b15 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.234719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.235366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.235653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.235777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d984b29-6876-4949-ab79-5b38df1a6e97 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.236050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.236757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:58.237130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.239076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.240467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.241119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.241401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.242130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.242356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:08:58.242655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.244231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.252571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 979/1963] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:08:58.256895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.260994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 985/1964] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:08:58.262008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.267348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3e89d-4fe1-43e0-b31e-07afdc2d0821 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.273991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.274615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.274895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.275004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77919c3e-f46a-42d7-abf6-414d928d2803 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.275165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.275857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.277915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.280062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.280724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.281039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.281324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.281992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.284497 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.310249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 980/1965] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:58.315179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.319561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 986/1966] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:58.324897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.395219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06545f-3cca-46ea-8ed4-323844a66fed admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.403468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded61f3-a27f-40fc-8de1-6577065bf556 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.409081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.409789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.410719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.411047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.411636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.412802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.414057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.414343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.414705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.414824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.415379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.419683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.452757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 981/1967] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:08:58.453338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 987/1968] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:08:58.457690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.457846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.468903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894b758-1435-4336-83db-cbe2665a5f2f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.470100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732aaf8-7cb9-492b-8a08-d572bd6ac555 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.475315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.476030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.476305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.476593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.477207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.477317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.477594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.477847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.478139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.478852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.479246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.480756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.481137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:58.492691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 988/1969] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:58.494230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 982/1970] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:08:58.497175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.499009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.510092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234316d-040b-41c9-85f5-e0ba7ab8c3f9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.510338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6e110-d13c-4d0e-afbc-55bf040c76aa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.515859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.516463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.516595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.516813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.517108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.517209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.517444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.517668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.517831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.518363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.520227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.520331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.538809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 989/1971] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:58.543754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 983/1972] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:08:58.543887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.545051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.559528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a0f7f-d2b9-4d01-bdfd-06c26a8565c1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.561156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a556d3d-5123-4b11-9dd9-a6a77568ef07 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.570370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.571192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.571610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.571948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.572090 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] PATH_INFO: `/v3/projects/c3457cffe0524d6f9402661bd2beaed9/users/9acd7d5dedbc43f9affbcb789abf462a/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.572709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.573247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.573523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] PATH_INFO: `/v3/projects/ca337984f6dc4f888cc6c68b0e9035d8/users/859f5e56c876498eb81038bb0d279692/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.592605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3457cffe0524d6f9402661bd2beaed9, user_id=9acd7d5dedbc43f9affbcb789abf462a, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.593390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca337984f6dc4f888cc6c68b0e9035d8, user_id=859f5e56c876498eb81038bb0d279692, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.595526 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416e34d-ebe6-4793-8bb3-e2fbb1bf44d3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.596616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed159d04-0b95-43b5-b12d-8252d3596c61 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.616165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 990/1973] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:58 2022] PUT /identity/v3/projects/c3457cffe0524d6f9402661bd2beaed9/users/9acd7d5dedbc43f9affbcb789abf462a/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:08:58.616804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 984/1974] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:08:58 2022] PUT /identity/v3/projects/ca337984f6dc4f888cc6c68b0e9035d8/users/859f5e56c876498eb81038bb0d279692/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:08:58.634853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf1a-1835-417c-a0ee-ed19d44af704 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.634853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf1a-1835-417c-a0ee-ed19d44af704 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.634853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf1a-1835-417c-a0ee-ed19d44af704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.635233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7d3c-4f90-4400-a050-3d18db798f25 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.637436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7d3c-4f90-4400-a050-3d18db798f25 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.637903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7d3c-4f90-4400-a050-3d18db798f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.653210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:08:58.671160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-eaec7d3c-4f90-4400-a050-3d18db798f25 None None] MFA Rules not processed for user `859f5e56c876498eb81038bb0d279692`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.672651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-eaec7d3c-4f90-4400-a050-3d18db798f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.679617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-29dfdf1a-1835-417c-a0ee-ed19d44af704 None None] MFA Rules not processed for user `9acd7d5dedbc43f9affbcb789abf462a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.680824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-29dfdf1a-1835-417c-a0ee-ed19d44af704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.781414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 985/1975] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:58.782714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.782999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.783278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.793711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 986/1976] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:08:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:08:58.794720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.796847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.803621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 991/1977] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:08:58.811960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.812599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.813007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.845281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:08:58.846713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:58.883150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e51f3-f483-42e7-bca9-6969bc8f8fb8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:58.887862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:58.888541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:58.888826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:58.889134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:58.899986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:58.901759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:58.919267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 987/1978] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:58.992620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:58.994927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:59.008450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 992/1979] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:08:59.031172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfba5c-b63e-46f5-a7bb-91bb5eda33fc service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:59.041114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:59.041760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:59.042041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:59.042625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:59.062379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:08:59.064233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:08:59.064344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:59.065786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:59.085009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009d308-62da-423f-a220-9cd50dbbd737 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:08:59.097999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:08:59.099245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:08:59.101601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:08:59.102224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 988/1980] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:08:59.104034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:08:59.133120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:08:59.133120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:08:59.144791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 993/1981] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:09:00.208854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:00.217212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38c438-65ef-4dae-b69d-244c1324b675 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:00.221448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:00.222123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:00.222407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:00.222682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:00.224668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:00.277995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:00.304156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:00.305644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db5dd4-30e6-495d-addb-5ecf95033d82 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:00.315513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 989/1982] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:09.137405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=67 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:10.311008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=12 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:23.536104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:23.537867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:23.609273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc811a-486c-43be-8a6d-393085e8bb06 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:23.615007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:23.615656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:23.615940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:23.616217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] PATH_INFO: `/v3/users/8d91476914e34fc8a39f750c3cdff93f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:23.617419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d91476914e34fc8a39f750c3cdff93f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:23.619423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:23.664420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d91476914e34fc8a39f750c3cdff93f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:23.676398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d91476914e34fc8a39f750c3cdff93f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:23.677097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d91476914e34fc8a39f750c3cdff93f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:23.683811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d91476914e34fc8a39f750c3cdff93f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:23.691124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d91476914e34fc8a39f750c3cdff93f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:23.697157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 994/1983] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:23 2022] DELETE /identity/v3/users/8d91476914e34fc8a39f750c3cdff93f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:23.701629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:23.702085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:23.702197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:23.720508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:23.721838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:23.787294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 990/1984] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:23.798260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:23.801308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:23.903573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39b57e-cb17-4502-8450-a9b0cd93e8af admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:23.907401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:23.908050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:23.908333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:23.908616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:23.919931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:23.921462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:23.932778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 995/1985] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:24.233497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:24.235130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.255344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62129676-e556-48e4-8373-25ffc18d4713 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:24.265928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:24.266862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:24.267375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:24.267857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] PATH_INFO: `/v3/users/859f5e56c876498eb81038bb0d279692` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:24.272365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=859f5e56c876498eb81038bb0d279692)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:24.275394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:24.356677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '859f5e56c876498eb81038bb0d279692'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.367148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '859f5e56c876498eb81038bb0d279692'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.386561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '859f5e56c876498eb81038bb0d279692'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.402250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '859f5e56c876498eb81038bb0d279692'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.403383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2f73ff85-a861-4af9-b15c-5791514a3e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '859f5e56c876498eb81038bb0d279692'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.407972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:24.409666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.418083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 991/1986] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:24 2022] DELETE /identity/v3/users/859f5e56c876498eb81038bb0d279692 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:24.419612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:24.420618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:24.420618 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:24.439197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c8c6-072e-4851-953e-67798b8f663e service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:24.444668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:24.446283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.464622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:24.516349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:24.517360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:24.517845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:24.518298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] PATH_INFO: `/v3/projects/e0d03f6f3f8b4086b8f52846edc4d5ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:24.519697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d03f6f3f8b4086b8f52846edc4d5ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:24.522006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:24.522842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 992/1987] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:24.534272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:24.535856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.556076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:24.592488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d03f6f3f8b4086b8f52846edc4d5ad', 'request_id': 'req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.601295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d03f6f3f8b4086b8f52846edc4d5ad', 'request_id': 'req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.608791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d03f6f3f8b4086b8f52846edc4d5ad', 'request_id': 'req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.632177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] The token cache is being invalidate because project e0d03f6f3f8b4086b8f52846edc4d5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:24.632872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.634211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.636458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 996/1988] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:24 2022] DELETE /identity/v3/projects/e0d03f6f3f8b4086b8f52846edc4d5ad => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:09:24.640844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:24.642891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.662600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b31e0-0095-4577-a38b-72bb98484809 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:24.749096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2a3d-a0f8-48eb-97a6-95dfc9f32012 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:24.756279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:24.757349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:24.757770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:24.758111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] PATH_INFO: `/v3/users/882ca5c6540240a988c08197c1715500` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:24.759753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882ca5c6540240a988c08197c1715500)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:24.762271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:24.782010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:24.783146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:24.783500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:24.785213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:24.788749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:24.817438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:24.819561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:24.840385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 993/1989] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:24.866784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882ca5c6540240a988c08197c1715500'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.878075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:24.892335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882ca5c6540240a988c08197c1715500'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.893814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882ca5c6540240a988c08197c1715500'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.901921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882ca5c6540240a988c08197c1715500'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.911100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882ca5c6540240a988c08197c1715500'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:24.920236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 997/1990] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:24 2022] DELETE /identity/v3/users/882ca5c6540240a988c08197c1715500 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:25.256522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.257466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.272590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71945e48-5195-4c49-b023-e5be348b1157 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.281464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.283168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.364564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.365398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.365581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.365978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] PATH_INFO: `/v3/projects/ca337984f6dc4f888cc6c68b0e9035d8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.367217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca337984f6dc4f888cc6c68b0e9035d8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.369636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.415056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2fbe3-65f0-491a-ac25-bbffe6688f14 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.443953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca337984f6dc4f888cc6c68b0e9035d8', 'request_id': 'req-e60d9482-a56f-41a5-ae84-86e28070189c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.451314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca337984f6dc4f888cc6c68b0e9035d8', 'request_id': 'req-e60d9482-a56f-41a5-ae84-86e28070189c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.459072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca337984f6dc4f888cc6c68b0e9035d8', 'request_id': 'req-e60d9482-a56f-41a5-ae84-86e28070189c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.475924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] The token cache is being invalidate because project ca337984f6dc4f888cc6c68b0e9035d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:25.476214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.476874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e60d9482-a56f-41a5-ae84-86e28070189c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.482170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 994/1991] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:25 2022] DELETE /identity/v3/projects/ca337984f6dc4f888cc6c68b0e9035d8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:25.484526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ced4-0c19-4537-a824-e80a61581c4b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.484950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ced4-0c19-4537-a824-e80a61581c4b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.485389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ced4-0c19-4537-a824-e80a61581c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.493502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:25.510518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0922ced4-0c19-4537-a824-e80a61581c4b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:25.512417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0922ced4-0c19-4537-a824-e80a61581c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.526006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.526810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.527162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.527483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] PATH_INFO: `/v3/projects/8369109497144a85a3b26a2c6be85cbc` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.529777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8369109497144a85a3b26a2c6be85cbc)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.532098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.599466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8369109497144a85a3b26a2c6be85cbc', 'request_id': 'req-6c8f1721-724b-49d3-b2b0-0920ae309712'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.605873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8369109497144a85a3b26a2c6be85cbc', 'request_id': 'req-6c8f1721-724b-49d3-b2b0-0920ae309712'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.608838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 995/1992] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:25.612917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8369109497144a85a3b26a2c6be85cbc', 'request_id': 'req-6c8f1721-724b-49d3-b2b0-0920ae309712'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.618232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585b71c-0151-48e9-b45b-08c1d3aea9a4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.618908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585b71c-0151-48e9-b45b-08c1d3aea9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.619761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585b71c-0151-48e9-b45b-08c1d3aea9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.626940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] The token cache is being invalidate because project 8369109497144a85a3b26a2c6be85cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:25.627233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.628457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6c8f1721-724b-49d3-b2b0-0920ae309712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:25.630342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 998/1993] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:25 2022] DELETE /identity/v3/projects/8369109497144a85a3b26a2c6be85cbc => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:25.635771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.636060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.636339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.653705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1585b71c-0151-48e9-b45b-08c1d3aea9a4 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:25.655208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1585b71c-0151-48e9-b45b-08c1d3aea9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.674339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:25.677320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.684351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:25.773820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 999/1994] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:25.774891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 996/1995] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:25.781360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.781948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.782235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.782784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.791530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4dc4e-7533-4a2f-ad63-a23b6e1fef6a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.796948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.797598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.798124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.798124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.798838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:25.799148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.801435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.804364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:25.805616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:25.814176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1000/1996] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:25.814472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 997/1997] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:25.819331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.819567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.829517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22077-0718-404e-9349-f4594ed7cd1f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.829723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e226-4dc5-4c72-b984-77e0d8cfdc06 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.834846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.835663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.835782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.836232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.836414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.836665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.836806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.837067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.837797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:25.837989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.838105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.839719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.840480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.854529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 998/1998] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:25.859646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.884389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da15a37-c889-42fc-abc5-1e1440d0ee35 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:25.891273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1001/1999] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:25.892646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.905451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:25.905451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:25.905451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:25.905451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:25.905748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:25.905748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:25.969162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 999/2000] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:09:25.971259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:25.993754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e26403-f248-4b65-823b-274372196d76 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.056130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc904c-8a4a-4113-834e-23c1c26f8e4f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.061777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.062561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.062954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.063695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.064505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.068458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.079051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:26.095270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1000/2001] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:26.099746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.109910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.110785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.114318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.114767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.118934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.123668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.125419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae661d3-5440-4488-b82f-8c0ce235920f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.142038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.142840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.143257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.143951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.144873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.150049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.166324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1001/2002] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:26.170457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.186424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1002/2003] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:25 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:26.191965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.204419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0dbff-d20f-490b-b13a-e67ced0ea5ca admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.206075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b7803-8e69-444b-88a0-8da0fd793fd6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.211380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.212010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.212298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.212670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.213420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.215354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.216161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.217155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.217765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.218371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.219207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.222383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.229417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:26.230555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1002/2004] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:26.236459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.239222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1003/2005] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:26.240758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.248752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b67c-0727-429f-a205-0cc22b11028f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.252057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185919e-bdf0-45d8-a506-680c2ac60d13 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.254322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.255027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.255351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.255628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] PATH_INFO: `/v3/projects/2e719196646743c69bb0d05e372a98b8/users/3a9a5512656a4d88997337afe4bcd2d6/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.262304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.263857 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.264336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.264742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.265953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.269725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.274087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e719196646743c69bb0d05e372a98b8, user_id=3a9a5512656a4d88997337afe4bcd2d6, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.277254 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc4f77a-c89c-4fe8-8f9a-c2ea0b06dbe2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.291094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1004/2006] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:09:26.292234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.300614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1003/2007] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:26 2022] PUT /identity/v3/projects/2e719196646743c69bb0d05e372a98b8/users/3a9a5512656a4d88997337afe4bcd2d6/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:09:26.307130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec4093-26c0-48bb-9cbc-3157bc391a79 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.313585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.313852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.314137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.366700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] MFA Rules not processed for user `3a9a5512656a4d88997337afe4bcd2d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:26.368631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:26.388653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.389605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.389925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.390308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] PATH_INFO: `/v3/projects/4d888dcf554944adae3e03bc1c24c52b/users/5fa85a53bfbe4ef5a36c01031f9016aa/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.410722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d888dcf554944adae3e03bc1c24c52b, user_id=5fa85a53bfbe4ef5a36c01031f9016aa, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.415056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff4f5c-fba7-4711-a792-d98211f2bfc0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.435222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1005/2008] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:26 2022] PUT /identity/v3/projects/4d888dcf554944adae3e03bc1c24c52b/users/5fa85a53bfbe4ef5a36c01031f9016aa/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:26.443638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.444017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.444813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.466658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1004/2009] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:26.469055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:26.470522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:26.482513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.485119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:26.548908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1006/2010] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:26.560226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.561709 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:26.566958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cec702-95c1-4717-9aef-4bfc624697f7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.633954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0f5a3-13b0-451f-9caa-47debb5804a8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.636490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.637318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.637707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.637821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.638373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.638487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.638655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.638932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.644999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:26.652216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.654172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.678538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1007/2011] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:09:26.725836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.727298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad074da-208c-46f9-b1c8-bd7f0c4c0cdd service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.770767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1005/2012] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:26.782620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:26.805659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a05144-fe5c-4b97-901c-021b3d7018b4 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:26.822289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:26.823081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:26.823363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:26.823639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] PATH_INFO: `/v3/users/3a9a5512656a4d88997337afe4bcd2d6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:26.825104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9a5512656a4d88997337afe4bcd2d6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:26.827655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:26.851710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:26.950627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9a5512656a4d88997337afe4bcd2d6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:26.986095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9a5512656a4d88997337afe4bcd2d6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:26.988337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9a5512656a4d88997337afe4bcd2d6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:27.005778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9a5512656a4d88997337afe4bcd2d6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:27.025356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9a5512656a4d88997337afe4bcd2d6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:27.042649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1008/2013] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:26 2022] DELETE /identity/v3/users/3a9a5512656a4d88997337afe4bcd2d6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:27.045174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:27.045643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:27.045817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:27.086211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:27.089020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:27.201370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1006/2014] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:27.208372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:27.210266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:27.383788 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396d82e-9009-4b4f-85e5-96588cc4a0a1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:27.399820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:27.400990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:27.401300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:27.401698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:27.424521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:27.428348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf654134-8061-48ce-947f-2edd8a0f5376 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:27.452026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1009/2015] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:28.738580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:28.740619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:28.765933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173e8a-3b24-4c04-8b2f-55e3d227276d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:28.812385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:28.812822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:28.813429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:28.813889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] PATH_INFO: `/v3/projects/2e719196646743c69bb0d05e372a98b8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:28.849258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e719196646743c69bb0d05e372a98b8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:28.869076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:28.949432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e719196646743c69bb0d05e372a98b8', 'request_id': 'req-dfee3cdc-06cf-4219-9800-4e602f51e9fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:28.961588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e719196646743c69bb0d05e372a98b8', 'request_id': 'req-dfee3cdc-06cf-4219-9800-4e602f51e9fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:28.980351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e719196646743c69bb0d05e372a98b8', 'request_id': 'req-dfee3cdc-06cf-4219-9800-4e602f51e9fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:29.017311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] The token cache is being invalidate because project 2e719196646743c69bb0d05e372a98b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:29.017587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:29.019111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfee3cdc-06cf-4219-9800-4e602f51e9fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:29.025052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1007/2016] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:28 2022] DELETE /identity/v3/projects/2e719196646743c69bb0d05e372a98b8 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:29.031442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.032020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.032151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.063032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:29.064898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:29.180654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1010/2017] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:29.188541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.188811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.189131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.229442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:29.230806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:29.263353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1008/2018] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:29.264624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.280577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bfb0a-c369-4425-971e-5b18b9e252ad None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.292180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.293547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.293921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.294274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.295408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:29.295988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.297811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.328058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1011/2019] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:29.329375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.341152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e4f1b-cc7a-43ee-bc29-8b7e2b47e9ee None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.348998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.350052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.350809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.351060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.351866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.356008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.394997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:29.411355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1009/2020] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/projects => generated 359 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:29.421399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.435604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448afd2-f0a6-44c0-a19c-dd9570799bb4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.460720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:29.545049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.545741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.546174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.546467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.547223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.551069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.590197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1012/2021] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/users => generated 280 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:29.594476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.615637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed4a00-bbb2-419e-b508-9b1c751e5899 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.622113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.622771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.623056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.623512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.624375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.626771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.660169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1010/2022] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:29.660606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.683392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7725d6-d8ad-4606-8ff1-e456d2168fcb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.699323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.700051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.702977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.703527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.704286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.710295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63885f-0e2c-4782-8f04-1a3599981e6f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.740193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1013/2023] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:29.744263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:29.764326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed406ac-9444-432b-b7bc-01d4f4c2b76b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:29.787084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:29.788180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.788531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.788875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] PATH_INFO: `/v3/projects/14dd66da34474fe8af1cf5a134c9dd60/users/4b2c715c7c3e497facbf7328c4f77b0c/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.820143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14dd66da34474fe8af1cf5a134c9dd60, user_id=4b2c715c7c3e497facbf7328c4f77b0c, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:29.824630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:29.847233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1011/2024] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:29 2022] PUT /identity/v3/projects/14dd66da34474fe8af1cf5a134c9dd60/users/4b2c715c7c3e497facbf7328c4f77b0c/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:29.856874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:29.857228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:29.857482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:29.929656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] MFA Rules not processed for user `4b2c715c7c3e497facbf7328c4f77b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:29.930906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.042380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1014/2025] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:30.079495 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.081490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.210829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97520c-8791-406f-9b56-bd602b0eed25 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.220108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.222085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.222728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.223367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.235602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.238249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ccb65-e901-45e6-afe5-45fd6c56b241 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.257572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1012/2026] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:30.274852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.445025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67df18f-08a0-497d-9271-931510b58e76 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.464608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.465290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.465573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.465847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] PATH_INFO: `/v3/users/4b2c715c7c3e497facbf7328c4f77b0c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.470026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b2c715c7c3e497facbf7328c4f77b0c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:30.474528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:30.526359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:30.564418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b2c715c7c3e497facbf7328c4f77b0c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:30.584707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b2c715c7c3e497facbf7328c4f77b0c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:30.585928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b2c715c7c3e497facbf7328c4f77b0c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:30.595604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b2c715c7c3e497facbf7328c4f77b0c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:30.605961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b2c715c7c3e497facbf7328c4f77b0c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:30.616242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1015/2027] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:30 2022] DELETE /identity/v3/users/4b2c715c7c3e497facbf7328c4f77b0c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:30.621529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.621798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.622082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:30.647466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:30.648718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.834271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1013/2028] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:30.854332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:30.855798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:30.987979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efa27c-bb36-4ff2-ab7c-635dcb5146f4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:30.991893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:30.992542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:30.992994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:30.993165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.007882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:31.009569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23a1c-fa93-4738-8ad4-f73f6cfa46a0 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:31.023670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1016/2029] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:31.817814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:31.819639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:31.863112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f824ea-e475-4a94-ad0d-81bed71e1943 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:31.879859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:31.881916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:31.882524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:31.883062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] PATH_INFO: `/v3/projects/14dd66da34474fe8af1cf5a134c9dd60` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:31.884769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14dd66da34474fe8af1cf5a134c9dd60)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:31.889023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.037093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14dd66da34474fe8af1cf5a134c9dd60', 'request_id': 'req-7fbc5b44-34d5-41e8-9869-2bead8dc919c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.061674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14dd66da34474fe8af1cf5a134c9dd60', 'request_id': 'req-7fbc5b44-34d5-41e8-9869-2bead8dc919c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.089553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14dd66da34474fe8af1cf5a134c9dd60', 'request_id': 'req-7fbc5b44-34d5-41e8-9869-2bead8dc919c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.113311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:32.138756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] The token cache is being invalidate because project 14dd66da34474fe8af1cf5a134c9dd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:32.141367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.143624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fbc5b44-34d5-41e8-9869-2bead8dc919c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:32.149650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1014/2030] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:31 2022] DELETE /identity/v3/projects/14dd66da34474fe8af1cf5a134c9dd60 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:32.154827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.155344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.155344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.195574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:32.196828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:32.366730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1017/2031] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:32.367952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.368241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.368607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.406827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:32.408317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:32.461802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1015/2032] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:32.467054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.482878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb24a97-7c9c-451a-ae8b-30500971bd0e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.495806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.496443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.496728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.497070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.497774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:32.498275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.499716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.540371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1018/2033] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:32.546078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.561908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b54d0-2c5d-42ee-ab67-4f494dab1ec6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.572326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.573309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.573602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.573884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.574573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.576533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.627095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1016/2034] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:32 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:32.629321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.640196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3018f-e107-4981-af00-dcc95308a79b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.702197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:32.745435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.746082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.746364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.746644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.747335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.749464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.779117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1019/2035] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:32 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:32.780341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.827208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7c5f1-ab9a-40c1-b511-1ce284a843d6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.835501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.836875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.837351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.838339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.839194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.841674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.853677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1017/2036] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:32.860993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.881850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac762a8-f86b-45b4-b033-88e732da21ec admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.889187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.889977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.890453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.890791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.891589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:32.894178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c2ecf-901b-4c93-95ee-b875d13be4fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:32.922232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1020/2037] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:32.926543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:32.946217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc774d86-0a5a-4df1-a9c7-be4a6b5811aa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:32.955921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:32.956945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:32.957325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:32.958365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] PATH_INFO: `/v3/projects/725810398afd433c9bff3fcd261b0f6f/users/f6aba59bcb664b8bbc1469d71b6a276d/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:32.988492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725810398afd433c9bff3fcd261b0f6f, user_id=f6aba59bcb664b8bbc1469d71b6a276d, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:33.006847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:33.036231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1018/2038] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:32 2022] PUT /identity/v3/projects/725810398afd433c9bff3fcd261b0f6f/users/f6aba59bcb664b8bbc1469d71b6a276d/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:33.046715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b1ada-0f91-4c9c-b3a6-ba3214b75a35 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:33.047208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b1ada-0f91-4c9c-b3a6-ba3214b75a35 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:33.047971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b1ada-0f91-4c9c-b3a6-ba3214b75a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:33.144986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-698b1ada-0f91-4c9c-b3a6-ba3214b75a35 None None] MFA Rules not processed for user `f6aba59bcb664b8bbc1469d71b6a276d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:33.146249 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-698b1ada-0f91-4c9c-b3a6-ba3214b75a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:33.286380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1021/2039] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:33.297843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:33.308183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:33.428052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d2d6b-ae75-4908-afd8-9050c0f52ce7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:33.432047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:33.432846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:33.433295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:33.433712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:33.445239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:33.447538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:33.472326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1019/2040] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:33.726993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:33.727286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:33.727564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:33.786735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] MFA Rules not processed for user `5fa85a53bfbe4ef5a36c01031f9016aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:33.788403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:33.868610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1022/2041] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:33.873247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:33.874984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:33.959112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2abe5-586f-4149-bbe7-075f3ad4b54c service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:33.963796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:33.964426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:33.964709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:33.965192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:33.965904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:33.968084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:33.995867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1020/2042] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:33 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:09:33.999879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:34.011854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa4e0c-9417-4081-a5f3-f0f7dbf30a26 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:34.041674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:34.069970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:34.070596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:34.070906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:34.071177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:34.071867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:34.073851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:34.095805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1023/2043] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:33 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:34.100432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:34.110602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7afbb-c329-46d2-8b25-83df19719dde admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:34.115339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:34.115944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:34.116225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:34.116502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:34.117237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:34.118867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:34.129310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1021/2044] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:09:34.133230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:34.143764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe224fb-a75e-44b1-80a1-0ce972217d6e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:34.148604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:34.149304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:34.149589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:34.149870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] PATH_INFO: `/v3/projects/bb6f73a4292d4ae4b7bd78a894936879/users/46fb4cbeed9b4927a35c8bade8dd12ab/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:34.165795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6f73a4292d4ae4b7bd78a894936879, user_id=46fb4cbeed9b4927a35c8bade8dd12ab, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:34.168871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:34.182978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1024/2045] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:34 2022] PUT /identity/v3/projects/bb6f73a4292d4ae4b7bd78a894936879/users/46fb4cbeed9b4927a35c8bade8dd12ab/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:34.186989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:34.193666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:34.198060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc915a-554f-4251-b50b-3d4b4b8b24ab admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:34.264508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:34.265172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:34.265483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:34.265756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:34.266453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:34.269148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc193d83-8662-40ee-a184-c184b6ebb6c3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:34.279678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1022/2046] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:34.284644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:34.295458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc086a22-c931-415f-8167-09b74e5557e5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:34.300361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:34.301054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:34.301337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:34.301616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] PATH_INFO: `/v3/projects/bb6f73a4292d4ae4b7bd78a894936879/users/46fb4cbeed9b4927a35c8bade8dd12ab/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:34.303581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6f73a4292d4ae4b7bd78a894936879, user_id=46fb4cbeed9b4927a35c8bade8dd12ab, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:34.306349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d565a48-898d-46e7-9746-d1bb8e03bca7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:34.318112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1025/2047] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:34 2022] PUT /identity/v3/projects/bb6f73a4292d4ae4b7bd78a894936879/users/46fb4cbeed9b4927a35c8bade8dd12ab/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:39.391373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:39.392472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:39.393018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:39.418273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] MFA Rules not processed for user `f6aba59bcb664b8bbc1469d71b6a276d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:39.419742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:39.499673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1023/2048] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:40.818192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a296-b678-4bb6-a447-11cd23fc50b6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:40.818692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a296-b678-4bb6-a447-11cd23fc50b6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:40.818915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a296-b678-4bb6-a447-11cd23fc50b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:40.890394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eeb5a296-b678-4bb6-a447-11cd23fc50b6 None None] MFA Rules not processed for user `46fb4cbeed9b4927a35c8bade8dd12ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:40.892125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eeb5a296-b678-4bb6-a447-11cd23fc50b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:41.023059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:41.024802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:41.031963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1026/2049] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:41.248731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56b7c7-109e-4b5b-8a23-e07548beb15f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:41.260523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:41.261636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:41.261932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:41.263538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:41.363523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:41.414445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:41.415923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:41.450790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1024/2050] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:41.775644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:41.776488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:41.777542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:41.799968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:41.801767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:41.905548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1027/2051] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:09:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:41.965243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:41.966674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:42.049924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a75874-db5a-4b48-8ca8-dc5d8e0308f1 service glance] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:42.060707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:42.061581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:42.061935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:42.062471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:42.078195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:42.081648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:42.093854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1025/2052] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:09:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:09:43.285371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:43.287568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:43.339679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:43.519488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdfda0-4906-4507-96cb-0c8f2d5bceac None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:43.539099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:43.539981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:43.540423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:43.542233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] PATH_INFO: `/v3/users/9acd7d5dedbc43f9affbcb789abf462a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:43.544006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acd7d5dedbc43f9affbcb789abf462a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:43.547680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:43.573486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:43.575489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:43.616698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789ead5-6cbc-4cef-a181-2664f3a8c3f1 service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:43.647447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acd7d5dedbc43f9affbcb789abf462a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.652200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:43.652880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:43.653192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:43.653472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] PATH_INFO: `/v3/users/215cb37035f8416e892dae268e4f93bc` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:43.660673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215cb37035f8416e892dae268e4f93bc)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:43.669453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:43.672804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acd7d5dedbc43f9affbcb789abf462a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.673663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acd7d5dedbc43f9affbcb789abf462a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.690337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acd7d5dedbc43f9affbcb789abf462a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.703520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-46d1b5d1-8a84-4abc-b67f-7093c277bf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acd7d5dedbc43f9affbcb789abf462a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.719206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1028/2053] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:43 2022] DELETE /identity/v3/users/9acd7d5dedbc43f9affbcb789abf462a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:09:43.725176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:43.725807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:43.728702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:43.764555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215cb37035f8416e892dae268e4f93bc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.782651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215cb37035f8416e892dae268e4f93bc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.785610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:43.789456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:43.815590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215cb37035f8416e892dae268e4f93bc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.841698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215cb37035f8416e892dae268e4f93bc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.843572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19d94c30-7cba-4494-a65c-1281b545ac70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215cb37035f8416e892dae268e4f93bc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:43.876045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1026/2054] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:43 2022] DELETE /identity/v3/users/215cb37035f8416e892dae268e4f93bc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:09:43.880784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:43.881557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:43.882368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:43.930317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:43.934329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:43.978614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1029/2055] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:44.004690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:44.007049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:44.025863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:44.079444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1027/2056] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:09:44.112814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:44.114669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:44.122335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:44.176478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac04246-d24d-426a-9ac7-c80191a5eb63 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:44.264726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae28ff-7114-42dc-b5ff-6218d5a2e928 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:44.270097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:44.270842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:44.271157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:44.271480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:44.284858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:44.287214 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:44.313132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1028/2057] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:09:44.321941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:44.323120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:44.323120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:44.323120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:44.350126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:44.351611 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3848b5-8e18-4555-a157-3813e4017e1e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:44.374726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1030/2058] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:09:45.030807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.034519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.065410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dd173-ed74-4c62-a283-84921237f7fe service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:45.084410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:45.087555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.097053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.097417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] PATH_INFO: `/v3/projects/c3457cffe0524d6f9402661bd2beaed9` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.100847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3457cffe0524d6f9402661bd2beaed9)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:45.102865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:45.228504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3457cffe0524d6f9402661bd2beaed9', 'request_id': 'req-e83559bc-6269-490b-b912-7db080ebc085'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.248723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3457cffe0524d6f9402661bd2beaed9', 'request_id': 'req-e83559bc-6269-490b-b912-7db080ebc085'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.264917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3457cffe0524d6f9402661bd2beaed9', 'request_id': 'req-e83559bc-6269-490b-b912-7db080ebc085'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.295188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] The token cache is being invalidate because project c3457cffe0524d6f9402661bd2beaed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:45.295503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.296394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.300865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1029/2059] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:45 2022] DELETE /identity/v3/projects/c3457cffe0524d6f9402661bd2beaed9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:45.306574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26cef3-e74e-459d-acb2-6318436ed9e1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.306861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26cef3-e74e-459d-acb2-6318436ed9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.307155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26cef3-e74e-459d-acb2-6318436ed9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.356703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.358290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.362230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5a26cef3-e74e-459d-acb2-6318436ed9e1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:45.366421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5a26cef3-e74e-459d-acb2-6318436ed9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.597331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83559bc-6269-490b-b912-7db080ebc085 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:45.612372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:45.613341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.613569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.613847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] PATH_INFO: `/v3/projects/f0092f02cc104f75a37174a216e728a2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.622354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1031/2060] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:45.631499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.631955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.633184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.644105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0092f02cc104f75a37174a216e728a2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:45.646036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:45.678303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:45.680700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:45.682563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.709313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1032/2061] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:45.710714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.728264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd6051-a5df-4862-9fe2-a3228d524152 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:45.741003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:45.741938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.742430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.742859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.743939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:45.745734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:45.748248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:45.773549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1033/2062] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:45.777069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.791432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ffe90-dac2-41d4-9d3b-30681aa1a38a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:45.796995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0092f02cc104f75a37174a216e728a2', 'request_id': 'req-9949b1f5-dac1-4e00-9d17-535085d4d5dd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.815662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0092f02cc104f75a37174a216e728a2', 'request_id': 'req-9949b1f5-dac1-4e00-9d17-535085d4d5dd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.824900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0092f02cc104f75a37174a216e728a2', 'request_id': 'req-9949b1f5-dac1-4e00-9d17-535085d4d5dd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.846751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] The token cache is being invalidate because project f0092f02cc104f75a37174a216e728a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:45.847804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.848662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:45.851648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1030/2063] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:45 2022] DELETE /identity/v3/projects/f0092f02cc104f75a37174a216e728a2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:45.856869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.861409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.883739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:45.884478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:45.884765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:45.885112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:45.885769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:45.887851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:45.915893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:45.928691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1034/2064] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:45 2022] POST /identity/v3/projects => generated 391 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:45.930060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:45.932871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:45.979829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949b1f5-dac1-4e00-9d17-535085d4d5dd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.011589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e076812-60ef-4d6a-8230-17f973c965a1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.017053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.018359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.018694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.019183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.019939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.023686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.054562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1035/2065] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:45 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:46.058672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:46.069494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.071177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.071614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.072210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] PATH_INFO: `/v3/users/730b179518bd48ebbf0ce689b942b406` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.073874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730b179518bd48ebbf0ce689b942b406)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.074683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417fb8-6e56-4262-9944-f1a8e02e8d9b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.076744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.088222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.089871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.089871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.089871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.090867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.093478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.113028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1036/2066] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:46.114321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:46.133159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42fb00-314e-421e-a5d9-ea4ce5f946db admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.147868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.148919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.149355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.149736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.150560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.152392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.166824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1037/2067] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:46.174976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730b179518bd48ebbf0ce689b942b406'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:46.181351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:46.182739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:46.189568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730b179518bd48ebbf0ce689b942b406'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:46.206842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e7b54-9155-4c5d-a1d2-9766d0028a22 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.217332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730b179518bd48ebbf0ce689b942b406'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:46.222068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.223197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.223564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.224211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] PATH_INFO: `/v3/projects/5015c8bab29c42da8ba7482c63abd257/users/df7f6c5b031e4c749b3325fcb069bac5/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.227274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730b179518bd48ebbf0ce689b942b406'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:46.228375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b6f06bfc-c6e0-4d88-88bc-12252c527ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730b179518bd48ebbf0ce689b942b406'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:46.238044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1031/2068] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:09:45 2022] DELETE /identity/v3/users/730b179518bd48ebbf0ce689b942b406 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:46.261771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5015c8bab29c42da8ba7482c63abd257, user_id=df7f6c5b031e4c749b3325fcb069bac5, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.265531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.318136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1038/2069] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:46 2022] PUT /identity/v3/projects/5015c8bab29c42da8ba7482c63abd257/users/df7f6c5b031e4c749b3325fcb069bac5/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:46.319222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.319521 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.319806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.395237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] MFA Rules not processed for user `df7f6c5b031e4c749b3325fcb069bac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:46.396498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:46.573879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1032/2070] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:46.585189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:46.586975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:46.698104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cdb2c-90c0-4713-ac50-952e2e749104 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:46.702403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:46.703047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:46.703319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:46.703596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:46.713119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:46.715041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf2692-64ce-4a9d-a077-742641c48a35 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:46.733439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1039/2071] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:47.169575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:47.172157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:47.359512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd18fc-67fb-41d1-b4a4-18547a830397 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:47.374110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:47.374859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:47.375084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:47.375360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] PATH_INFO: `/v3/projects/445fc7fd0ae14328b6a014d8489d7400` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:47.378880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445fc7fd0ae14328b6a014d8489d7400)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:47.378880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:47.476733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445fc7fd0ae14328b6a014d8489d7400', 'request_id': 'req-2cfb52d4-773e-46de-b9d1-20e0d283ece4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:47.482973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=1,keystone:DELETE=9,keystone:SELECT=90 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:47.484425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445fc7fd0ae14328b6a014d8489d7400', 'request_id': 'req-2cfb52d4-773e-46de-b9d1-20e0d283ece4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:47.492736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445fc7fd0ae14328b6a014d8489d7400', 'request_id': 'req-2cfb52d4-773e-46de-b9d1-20e0d283ece4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:47.510133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] The token cache is being invalidate because project 445fc7fd0ae14328b6a014d8489d7400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:09:47.510631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:47.511652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2cfb52d4-773e-46de-b9d1-20e0d283ece4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:09:47.513771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1033/2072] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:09:47 2022] DELETE /identity/v3/projects/445fc7fd0ae14328b6a014d8489d7400 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:47.520927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:47.521292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:47.521573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:47.552758 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:47.554533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:47.674321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:47.706643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1040/2073] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:47.709874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:47.710166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:47.710444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:47.746979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:47.748377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:47.760181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1034/2074] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:47.765972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:47.788258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee52c8a-6e23-4f09-9e25-eb9c188cabb2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:47.795506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:47.796265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:47.796614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:47.796956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:47.816030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:09:47.820215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:47.823152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:47.855212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:47.879614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1041/2075] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:09:47.880692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c420d-6909-44ce-bc25-3ea4a7059c44 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:47.886828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:47.887562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:47.887976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:47.888365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:47.889298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:47.896304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:47.937499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:47.941037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1035/2076] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:47 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:47.950617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee6a17-80d3-4018-a661-3297af586f20 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.022564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.022564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.022926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.023122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.023809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.025796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.055755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1042/2077] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:47 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:09:48.057293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.068994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29827b-ba5b-423a-bd56-86262520952b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.079645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.079645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.079645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.079645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.080612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.082203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.097559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1036/2078] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:48.101866 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.126403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844b9b3-4097-4983-ba58-38a83c81781a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.137128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.138021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.138460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.138894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.139743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.141801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f3f61-aa58-48f0-9a03-37a5bf83d45d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.160121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1043/2079] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:48.164004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.174601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6812-49bc-4748-92c8-57212aeaa397 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.179505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.180380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.180696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.181166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] PATH_INFO: `/v3/projects/f6ab745349dd41aab6b2ad4f7563a362/users/8686ed78c6dc4940839dd6e267135121/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.220155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ab745349dd41aab6b2ad4f7563a362, user_id=8686ed78c6dc4940839dd6e267135121, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.229151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.247724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1037/2080] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:48 2022] PUT /identity/v3/projects/f6ab745349dd41aab6b2ad4f7563a362/users/8686ed78c6dc4940839dd6e267135121/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:48.258695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.259279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.259566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.318903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] MFA Rules not processed for user `8686ed78c6dc4940839dd6e267135121`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:48.318903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:48.499202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1044/2081] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:48.505131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.672270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e485ab-5e4a-4785-876f-4118ef4d304c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.681856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.684276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.684567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.684926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.690020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.692156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.783495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.784929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1038/2082] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:48 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:48.804072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68433b-257a-4593-985d-28ae0027cef7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:48.889682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:48.947833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:48.948565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:48.949056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:48.949430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:48.950207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:48.952402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:48.982734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1045/2083] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:48 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:09:48.988103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:48.999212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095f75c-6bb1-4e13-bcaf-d63134218911 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.008695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.009376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.009657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.009928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.010635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.012256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.049257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1039/2084] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:49.049474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.060067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92d12-2772-48e0-97aa-63073edeca22 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.077372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.077577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.077860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.078133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.080794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.082475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb99b4-f711-491b-be37-8aeaf4131661 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.103656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1046/2085] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:49.105130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.118206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91385ce-0517-46b2-af0d-ceb8479fddc3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.123773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.125036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.126140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.128494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] PATH_INFO: `/v3/projects/24553759637e4aeaa6b375b9a51e0df6/users/be32f31c2e404432bd472feb7d02f20d/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.147146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=24553759637e4aeaa6b375b9a51e0df6, user_id=be32f31c2e404432bd472feb7d02f20d, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.150802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.171159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1040/2086] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:49 2022] PUT /identity/v3/projects/24553759637e4aeaa6b375b9a51e0df6/users/be32f31c2e404432bd472feb7d02f20d/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:49.182746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.183426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.184076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.242301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] MFA Rules not processed for user `be32f31c2e404432bd472feb7d02f20d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:49.246178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:49.371854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1047/2087] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:49.376179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.440425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:49.484008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce613fc-b717-4093-95a7-f238811a32fc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.490977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.491698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.492047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.492328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.493054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.494979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.527372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1041/2088] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:09:49 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:49.531703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.547104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97a7da-b236-4a57-9189-51131794b6e7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.631444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.632061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.632352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.632621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.633319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.637829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.665325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1048/2089] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:09:49 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:09:49.667018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.679156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ca3e-a2fc-4089-bf8f-f2a8b248a104 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.688886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.690231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.690704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.690990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.692054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.694011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.711923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1042/2090] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:49.716211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.730448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b4bab-ca10-4508-9d1e-685a30e0284e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.735864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.736513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.737549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.737549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] PATH_INFO: `/v3/projects/704b006b41b940deacf0ba5499ad6358/users/bbefe654401b4a368729a9f5ab7a0efa/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.755260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704b006b41b940deacf0ba5499ad6358, user_id=bbefe654401b4a368729a9f5ab7a0efa, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.758009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.777017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1049/2091] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:49 2022] PUT /identity/v3/projects/704b006b41b940deacf0ba5499ad6358/users/bbefe654401b4a368729a9f5ab7a0efa/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:49.780243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.795538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737806d-430e-4144-8231-b435b11f93b1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.881391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.882108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.882451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.882738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.883526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.885256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64c449-e33b-4cee-ba6c-f95ee723082e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.897034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1043/2092] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:09:49.900606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:49.913720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddd405-d593-4609-8437-8556050aece1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:49.921474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:49.922472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.922915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.923369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] PATH_INFO: `/v3/projects/704b006b41b940deacf0ba5499ad6358/users/bbefe654401b4a368729a9f5ab7a0efa/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.926214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704b006b41b940deacf0ba5499ad6358, user_id=bbefe654401b4a368729a9f5ab7a0efa, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:49.929128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:49.945741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1050/2093] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:09:49 2022] PUT /identity/v3/projects/704b006b41b940deacf0ba5499ad6358/users/bbefe654401b4a368729a9f5ab7a0efa/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:09:49.958030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:49.958736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:49.959398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:49.991282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] MFA Rules not processed for user `bbefe654401b4a368729a9f5ab7a0efa`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:49.992479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:50.102468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1044/2094] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:09:50.113643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:50.115650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:50.121317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:50.195966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40024485-6908-4b37-b690-9bcc4bc30568 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:50.202284 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:50.203087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:50.203374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:50.203650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:50.213977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:50.215764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118db60f-cfc3-4753-9d27-a5e9468241bf service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:50.227053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1051/2095] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:09:50.891527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:09:50.894524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:50.915158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9584e-e887-495b-8aa6-a8c3bb0847d3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:09:50.919482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:09:50.920114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:50.920399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:50.920680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:50.952566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:09:50.999098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:09:51.000557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:09:51.014906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1045/2096] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:09:55.248499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:09:55.249056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:09:55.249168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:09:55.268211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] MFA Rules not processed for user `5fa85a53bfbe4ef5a36c01031f9016aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:09:55.269374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:09:55.324141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1052/2097] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:09:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:01.006976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=16 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:03.708998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:03.718858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645198-ab6f-4289-91e0-bc94d14f1a7a service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:03.723591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:03.724312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:03.724486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:03.724766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:03.788738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:03.792076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f9528-5b6f-45a7-ba52-0906b74cecab service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:03.804015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1046/2098] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:05.316779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=50 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:10.096427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:10.177852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81209768-03ee-4dfc-af19-71bf99a6a6ad None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:10.182736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:10.183370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:10.183649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:10.183923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] PATH_INFO: `/v3/users/8686ed78c6dc4940839dd6e267135121` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:10.185110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8686ed78c6dc4940839dd6e267135121)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:10.187029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:10.249451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8686ed78c6dc4940839dd6e267135121'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:10.262889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8686ed78c6dc4940839dd6e267135121'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:10.263605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8686ed78c6dc4940839dd6e267135121'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:10.270380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8686ed78c6dc4940839dd6e267135121'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:10.278219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8686ed78c6dc4940839dd6e267135121'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:10.286173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1053/2099] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:10 2022] DELETE /identity/v3/users/8686ed78c6dc4940839dd6e267135121 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:10.290722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2e606-2125-4bfc-8643-96ce2a06cba8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:10.291165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2e606-2125-4bfc-8643-96ce2a06cba8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:10.291441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2e606-2125-4bfc-8643-96ce2a06cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:10.311196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-61e2e606-2125-4bfc-8643-96ce2a06cba8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:10.312508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-61e2e606-2125-4bfc-8643-96ce2a06cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:10.403043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1047/2100] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:10.410746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:10.412339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:10.510024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb5acd-9258-4ef1-b9ef-41caedb7c228 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:10.513917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:10.514642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:10.514998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:10.515331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:10.531255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:10.532695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:10.543989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1054/2101] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:11.029566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:11.030164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:11.030667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:11.060233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:11.061898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:11.146042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1048/2102] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:10:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:11.161130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:11.163569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:11.198304 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:11.270680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938752e4-e5fd-49b5-8bc3-be16152743c8 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:11.287096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:11.287894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:11.288232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:11.288516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:11.302236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:11.304147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:11.314877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1055/2103] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 20:10:11.353798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:11.357489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:11.402748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e5d89-52e2-4663-97f2-6d7476c9744b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:11.405800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:11.406483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:11.406940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:11.407393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] PATH_INFO: `/v3/projects/f6ab745349dd41aab6b2ad4f7563a362` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:11.417699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ab745349dd41aab6b2ad4f7563a362)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:11.422144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:11.522018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:11.535866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ab745349dd41aab6b2ad4f7563a362', 'request_id': 'req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.544329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ab745349dd41aab6b2ad4f7563a362', 'request_id': 'req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.555218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ab745349dd41aab6b2ad4f7563a362', 'request_id': 'req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.604993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] The token cache is being invalidate because project f6ab745349dd41aab6b2ad4f7563a362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:11.605289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.605993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.607932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1049/2104] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:11 2022] DELETE /identity/v3/projects/f6ab745349dd41aab6b2ad4f7563a362 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:11.613678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:11.628519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d240d-8c0d-4cc5-9ad2-359c0a1df830 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:11.801078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:11.801739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:11.802117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:11.802281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] PATH_INFO: `/v3/users/be32f31c2e404432bd472feb7d02f20d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:11.803455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be32f31c2e404432bd472feb7d02f20d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:11.805592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:11.895940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be32f31c2e404432bd472feb7d02f20d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.921117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be32f31c2e404432bd472feb7d02f20d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.922312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be32f31c2e404432bd472feb7d02f20d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.930518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be32f31c2e404432bd472feb7d02f20d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.944742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be32f31c2e404432bd472feb7d02f20d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:11.956649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1056/2105] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:11 2022] DELETE /identity/v3/users/be32f31c2e404432bd472feb7d02f20d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:12.263540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:12.264997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:12.283449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2865c7c-4bd3-4cf1-ba6c-c9f8f87c8a52 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:12.345363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:12.346032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:12.346310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:12.346583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] PATH_INFO: `/v3/projects/24553759637e4aeaa6b375b9a51e0df6` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:12.347764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24553759637e4aeaa6b375b9a51e0df6)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:12.349705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:12.397168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24553759637e4aeaa6b375b9a51e0df6', 'request_id': 'req-f6500731-7c40-4c31-8efe-a268b309c515'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.402511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24553759637e4aeaa6b375b9a51e0df6', 'request_id': 'req-f6500731-7c40-4c31-8efe-a268b309c515'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.408853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24553759637e4aeaa6b375b9a51e0df6', 'request_id': 'req-f6500731-7c40-4c31-8efe-a268b309c515'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.421771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] The token cache is being invalidate because project 24553759637e4aeaa6b375b9a51e0df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:12.422070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.422703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.424847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1050/2106] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:12 2022] DELETE /identity/v3/projects/24553759637e4aeaa6b375b9a51e0df6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:12.428953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:12.430468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:12.448456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7c51-5921-4d35-8471-fb036ee5708d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:12.479370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:12.510686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:12.511342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:12.511625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:12.511929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] PATH_INFO: `/v3/users/bbefe654401b4a368729a9f5ab7a0efa` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:12.513158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbefe654401b4a368729a9f5ab7a0efa)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:12.515118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:12.562766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbefe654401b4a368729a9f5ab7a0efa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.575066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbefe654401b4a368729a9f5ab7a0efa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.575749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbefe654401b4a368729a9f5ab7a0efa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.582623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbefe654401b4a368729a9f5ab7a0efa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.589844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8d849f68-9ad4-4c75-b9fa-c9245f552a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbefe654401b4a368729a9f5ab7a0efa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:12.595676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1057/2107] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:12 2022] DELETE /identity/v3/users/bbefe654401b4a368729a9f5ab7a0efa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:13.128470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.130017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:13.151574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500731-7c40-4c31-8efe-a268b309c515 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.257012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:13.263630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.264617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.265017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.265305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] PATH_INFO: `/v3/projects/704b006b41b940deacf0ba5499ad6358` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.266700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] RBAC: Authorizing `identity:delete_project(project_id=704b006b41b940deacf0ba5499ad6358)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.269300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.314646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704b006b41b940deacf0ba5499ad6358', 'request_id': 'req-f87f6335-60a9-4c81-beb1-55836083ceee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:13.320432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704b006b41b940deacf0ba5499ad6358', 'request_id': 'req-f87f6335-60a9-4c81-beb1-55836083ceee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:13.327021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704b006b41b940deacf0ba5499ad6358', 'request_id': 'req-f87f6335-60a9-4c81-beb1-55836083ceee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:13.340108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] The token cache is being invalidate because project 704b006b41b940deacf0ba5499ad6358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:13.340400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:13.341049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f87f6335-60a9-4c81-beb1-55836083ceee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:13.342617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1051/2108] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:13 2022] DELETE /identity/v3/projects/704b006b41b940deacf0ba5499ad6358 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:13.349256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.349548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.349826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.371046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:13.372548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:13.433436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1058/2109] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:13.444604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.445186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.445379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.467847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:13.469116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:13.478023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1052/2110] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:13.482374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.493010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cc3f1-fbbc-4a6c-86b7-5b1dea4e294d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.498605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.499491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.499726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.500101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.500948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:10:13.501278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.503694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.521315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1059/2111] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:10:13.523230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.534383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d212-8399-4031-9100-27ed00d28af2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.539761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.540390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.540679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.541107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.541873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.544160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.584178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1053/2112] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:13.585154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.597621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692262f-2138-4de3-8e1d-79b167ed2724 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.636546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:13.663306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.663944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.664230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.664509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.665234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.667242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.698679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1060/2113] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/users => generated 305 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:10:13.700190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.711760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff78b08-8099-4e71-889b-419c552b4e89 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.716713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.717351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.717636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.717916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.718612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.720242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.731592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1054/2114] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:13.735613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.746958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f00a7-3964-4e75-8b69-c4bda43ed884 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.753840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.754591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.755679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.755966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.756691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.758633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d8363-acc5-48a6-a8a4-0dd6b041fa17 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.773147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1061/2115] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:13.776884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:13.799293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb28e9-87c5-43e9-a25f-a95dbc7bc3da admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:13.808986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:13.809681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.809965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.810246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] PATH_INFO: `/v3/projects/3b33e85b497949f799d6902953421a6e/users/60d9231e770f43b4923b6432821eab90/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.826815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b33e85b497949f799d6902953421a6e, user_id=60d9231e770f43b4923b6432821eab90, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:13.829545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:13.854315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1055/2116] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:13 2022] PUT /identity/v3/projects/3b33e85b497949f799d6902953421a6e/users/60d9231e770f43b4923b6432821eab90/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:13.858295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:13.858567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:13.858845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:13.888113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] MFA Rules not processed for user `60d9231e770f43b4923b6432821eab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:13.889489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:13.973628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1062/2117] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:13.980413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.058773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d121ce8-80e3-4df6-9ec4-362c738c3ecc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.065986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.067179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.067575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.068758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.075643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.097798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.169249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1056/2118] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:13 2022] POST /identity/v3/projects => generated 407 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:14.174079 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.205018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116189f-7cc0-4937-8ab4-04658a611e35 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.326643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.327385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.327674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.328256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.330118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.337378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.367644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1063/2119] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:14 2022] POST /identity/v3/users => generated 310 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:14.372827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.386854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a655-fc7a-496f-9b5f-7a715ee1a9dc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.391982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.392699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.393670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.394018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.394801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.396513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.399066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:14.409924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1057/2120] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:14.419201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.438926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfefd11-0dc6-4b6b-a2ce-215196968703 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.445052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.445846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.446235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.446586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] PATH_INFO: `/v3/projects/b9d589cb876e4a97860ae034f96d5245/users/20490e2bbcad4782a7385bb131bca386/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.463659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d589cb876e4a97860ae034f96d5245, user_id=20490e2bbcad4782a7385bb131bca386, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.466824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.481112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1064/2121] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:14 2022] PUT /identity/v3/projects/b9d589cb876e4a97860ae034f96d5245/users/20490e2bbcad4782a7385bb131bca386/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:14.485225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.497180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d8536-63de-4e78-a703-ca32f5b78eb5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.570064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.570950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.571233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.571514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.572217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.574248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de242e-a51f-439b-a530-61df1fc33253 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.584633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1058/2122] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:14.588888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.599521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d010f9-2ee1-46d6-a783-4562db1226d6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.604669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.605469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.605850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.606131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] PATH_INFO: `/v3/projects/b9d589cb876e4a97860ae034f96d5245/users/20490e2bbcad4782a7385bb131bca386/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.608430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d589cb876e4a97860ae034f96d5245, user_id=20490e2bbcad4782a7385bb131bca386, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.611358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.618773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:14.624944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1065/2123] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:14 2022] PUT /identity/v3/projects/b9d589cb876e4a97860ae034f96d5245/users/20490e2bbcad4782a7385bb131bca386/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:14.637039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744470b-3701-4284-8f7f-987d0649318e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.637672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744470b-3701-4284-8f7f-987d0649318e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.637868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744470b-3701-4284-8f7f-987d0649318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.676946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3744470b-3701-4284-8f7f-987d0649318e None None] MFA Rules not processed for user `20490e2bbcad4782a7385bb131bca386`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:14.678213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3744470b-3701-4284-8f7f-987d0649318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:14.748466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1059/2124] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:14 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:14.772655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:14.774583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:14.848538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f18a7-8375-44e9-975e-842296324528 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:14.852359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:14.853010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:14.853299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:14.853578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:14.864630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:14.866456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:14.879266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1066/2125] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:14 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:15.068451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744470b-3701-4284-8f7f-987d0649318e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:15.084059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744470b-3701-4284-8f7f-987d0649318e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:15.093162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:15.093905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:15.094193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:15.094479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:15.166830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:15.168282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd893f9-17cb-4bec-b954-8dac01e98252 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:15.177857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1060/2126] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:15.295792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:15.298365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:15.435655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a96e4-c3b5-4f22-b667-6bbb278d9b3b service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:15.441375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:15.442256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:15.442632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:15.443334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] PATH_INFO: `/v3/users/df7f6c5b031e4c749b3325fcb069bac5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:15.444733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7f6c5b031e4c749b3325fcb069bac5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:15.446936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:15.547499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7f6c5b031e4c749b3325fcb069bac5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:15.582586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7f6c5b031e4c749b3325fcb069bac5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:15.583187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7f6c5b031e4c749b3325fcb069bac5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:15.597078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7f6c5b031e4c749b3325fcb069bac5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:15.629930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7f6c5b031e4c749b3325fcb069bac5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:15.645956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1067/2127] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:15 2022] DELETE /identity/v3/users/df7f6c5b031e4c749b3325fcb069bac5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:15.650786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:15.650786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:15.651256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:15.673393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:15.673952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:15.697682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:15.751207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1061/2128] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:15.758680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:15.760127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:15.785065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:15.837879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfaa62-f042-4f95-b92b-17d70e3fa34c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:15.841541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:15.842157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:15.842437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:15.842712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:15.853283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:15.854743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:15.865856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1068/2129] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:16.212861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:16.214761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:16.236073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3966cbf-af18-4c14-8d25-ad9bfded7985 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:16.242070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:16.243040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:16.243424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:16.243800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] PATH_INFO: `/v3/projects/5015c8bab29c42da8ba7482c63abd257` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:16.245115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5015c8bab29c42da8ba7482c63abd257)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:16.247156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:16.302891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5015c8bab29c42da8ba7482c63abd257', 'request_id': 'req-7cb90f46-6942-4998-8885-a142aa4d032a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:16.308625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5015c8bab29c42da8ba7482c63abd257', 'request_id': 'req-7cb90f46-6942-4998-8885-a142aa4d032a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:16.315280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5015c8bab29c42da8ba7482c63abd257', 'request_id': 'req-7cb90f46-6942-4998-8885-a142aa4d032a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:16.327811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] The token cache is being invalidate because project 5015c8bab29c42da8ba7482c63abd257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:16.328101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:16.328741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7cb90f46-6942-4998-8885-a142aa4d032a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:16.330889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1062/2130] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:16 2022] DELETE /identity/v3/projects/5015c8bab29c42da8ba7482c63abd257 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:25.860285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=17 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:26.325085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:28.089220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:28.090868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:28.107329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e6fc6-9bee-4047-ade5-83c77eb8ed39 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:28.193367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:28.193960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:28.194248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:28.194525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] PATH_INFO: `/v3/users/60d9231e770f43b4923b6432821eab90` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:28.195707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d9231e770f43b4923b6432821eab90)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:28.197688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:28.244125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d9231e770f43b4923b6432821eab90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.256384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d9231e770f43b4923b6432821eab90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.257114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d9231e770f43b4923b6432821eab90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.264162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d9231e770f43b4923b6432821eab90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.276337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d9231e770f43b4923b6432821eab90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.284876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1069/2131] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:28 2022] DELETE /identity/v3/users/60d9231e770f43b4923b6432821eab90 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:28.289909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:28.290372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:28.290486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:28.309626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:28.310842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:28.374213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1063/2132] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:28.384666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:28.386121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:28.451334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed667f47-a3f2-47c8-881f-43c02da410b1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:28.456073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:28.457125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:28.457406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:28.457688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:28.467270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:28.468702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:28.479279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1070/2133] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:10:28.748337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:28.749827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:28.762973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7aa8b-6ce9-4c6e-ab1c-c7e69b64f785 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:28.767564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:28.768195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:28.768488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:28.768760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] PATH_INFO: `/v3/projects/3b33e85b497949f799d6902953421a6e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:28.770071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b33e85b497949f799d6902953421a6e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:28.772114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:28.821589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b33e85b497949f799d6902953421a6e', 'request_id': 'req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.829021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b33e85b497949f799d6902953421a6e', 'request_id': 'req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.837033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b33e85b497949f799d6902953421a6e', 'request_id': 'req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.854452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] The token cache is being invalidate because project 3b33e85b497949f799d6902953421a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:28.854747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.855667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:28.857326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1064/2134] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:28 2022] DELETE /identity/v3/projects/3b33e85b497949f799d6902953421a6e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:28.861838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:28.877929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389701-5017-48d4-9d11-c29b590575c4 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:28.904413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:28.987511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:28.988180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:28.988467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:28.988755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] PATH_INFO: `/v3/users/20490e2bbcad4782a7385bb131bca386` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:28.989998 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20490e2bbcad4782a7385bb131bca386)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:28.991954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:29.054300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20490e2bbcad4782a7385bb131bca386'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.068599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20490e2bbcad4782a7385bb131bca386'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.069724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20490e2bbcad4782a7385bb131bca386'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.083895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20490e2bbcad4782a7385bb131bca386'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.091494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4b9bc5d5-9c95-4af3-9654-c5563fb5a34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20490e2bbcad4782a7385bb131bca386'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.098563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1071/2135] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:28 2022] DELETE /identity/v3/users/20490e2bbcad4782a7385bb131bca386 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:29.471786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:29.474764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:29.491767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3355-acde-4cfc-8e6c-5eaa2a758c6e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:29.571939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:29.572681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.573683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.574020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] PATH_INFO: `/v3/projects/b9d589cb876e4a97860ae034f96d5245` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:29.575293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d589cb876e4a97860ae034f96d5245)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:29.577385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:29.619960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:29.647627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d589cb876e4a97860ae034f96d5245', 'request_id': 'req-c5ef8eef-c305-4722-ac2d-e1aad92a3369'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.657532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d589cb876e4a97860ae034f96d5245', 'request_id': 'req-c5ef8eef-c305-4722-ac2d-e1aad92a3369'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.664927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d589cb876e4a97860ae034f96d5245', 'request_id': 'req-c5ef8eef-c305-4722-ac2d-e1aad92a3369'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.680924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] The token cache is being invalidate because project b9d589cb876e4a97860ae034f96d5245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:29.681425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.682143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c5ef8eef-c305-4722-ac2d-e1aad92a3369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:29.684410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1065/2136] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:29 2022] DELETE /identity/v3/projects/b9d589cb876e4a97860ae034f96d5245 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:29.691276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.691637 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.691903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:29.715644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:29.717332 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:29.788379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1072/2137] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:29.795998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.796294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.796575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:29.817188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:29.818657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:29.826629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1066/2138] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:29.830978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:29.840756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4f047-680b-43e4-b155-8bb3b320cbd7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:29.845407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:29.846185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.846475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.846753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:29.847450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:10:29.847763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:29.849406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:29.860610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1073/2139] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:10:29.865282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:29.875262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef634f-6d32-4088-b063-abf20b8eeb13 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:29.880894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:29.881536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.881821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.882096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:29.882754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:29.884664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:29.913630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1067/2140] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:29 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:29.917776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:29.930294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9389ddc-6e19-43f3-a8f4-33be5c7763b3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:29.964505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:29.998599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:29.999231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:29.999548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:29.999781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.000469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.003072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.029324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1074/2141] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:29 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:30.030546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.041123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948b51c-c42d-4c06-84d4-0b906bfd68e3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.045843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.046460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.046742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.047018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.047717 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.049453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.061456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1068/2142] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:30.066142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.079898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84833-798a-48de-88a9-2fc367b6341f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.102226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.103153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.108200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.108200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.108200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.110065 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a6c75-4847-42f8-ade5-1dd947eee5c9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.131530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1075/2143] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:30.135449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.146459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d93136-fc6d-4d1a-82b0-dfd958e0cb6e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.151776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.152530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.152880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.153257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] PATH_INFO: `/v3/projects/7b1d877f14b24c63b5c7af23f0568e14/users/191f0d516dbf422dafbb1dea14cf405e/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.170208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1d877f14b24c63b5c7af23f0568e14, user_id=191f0d516dbf422dafbb1dea14cf405e, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.172984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.186941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1069/2144] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:30 2022] PUT /identity/v3/projects/7b1d877f14b24c63b5c7af23f0568e14/users/191f0d516dbf422dafbb1dea14cf405e/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:30.198892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.199179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.199463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.228481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] MFA Rules not processed for user `191f0d516dbf422dafbb1dea14cf405e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:30.229769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:30.296069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1076/2145] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:30.301336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.362858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92275c5-8f90-4cca-b570-6658d19de09a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.367414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.368021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.368300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.368576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.369267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.371159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.397430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1070/2146] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:30 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:30.401676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.413258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c67c-0a28-420e-93d3-2ce75ceef652 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.471785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.472411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.472698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.473084 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.473670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.475599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.500568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1077/2147] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:30 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:30.505830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.517018 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc98d2e-b0a2-48b5-af57-ba1cb26b80e9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.521985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.522610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.522891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.523163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.524080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.525720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.540168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1071/2148] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:30.541477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.554443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9461c71-1ee0-439c-9e11-d87185720372 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.559483 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.560158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.560442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.560721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] PATH_INFO: `/v3/projects/1c84c5172374433292b2c3e93602c355/users/ef40d1456e1b415c9779c251d694ed46/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.587635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c84c5172374433292b2c3e93602c355, user_id=ef40d1456e1b415c9779c251d694ed46, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.590704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.612682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1078/2149] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:30 2022] PUT /identity/v3/projects/1c84c5172374433292b2c3e93602c355/users/ef40d1456e1b415c9779c251d694ed46/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:30.616800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.635246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86022079-ce02-4494-a0d1-f1da4577aceb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.676512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:30.729831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.730353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.730642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.730920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.731819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.733474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ea915-fcf1-4f0d-a397-5987c942925f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.751498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.751759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1072/2150] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:30.758464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:30.766072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28371f18-14bd-488a-b2c9-99b549311cb6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:30.773153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:30.773851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.774192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.774528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] PATH_INFO: `/v3/projects/1c84c5172374433292b2c3e93602c355/users/ef40d1456e1b415c9779c251d694ed46/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.782913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c84c5172374433292b2c3e93602c355, user_id=ef40d1456e1b415c9779c251d694ed46, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:30.787229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:30.803649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1079/2151] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:30 2022] PUT /identity/v3/projects/1c84c5172374433292b2c3e93602c355/users/ef40d1456e1b415c9779c251d694ed46/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:30.814969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:30.815477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:30.815866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:30.845339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] MFA Rules not processed for user `ef40d1456e1b415c9779c251d694ed46`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:30.847387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:30.939126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1073/2152] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:30.951159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:30.952850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:31.014155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e109236-0330-4637-99c2-d0ece115f15e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:31.018817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:31.019427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:31.019704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:31.019979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:31.117404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:31.119264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd92b38-dc6c-4fbf-aeb7-7376e01f6083 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:31.134942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1080/2153] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:34.816341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:34.904880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5db89-519f-4a2f-8259-11222a5754bc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:34.912178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:34.913011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:34.913572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:34.913708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] PATH_INFO: `/v3/users/191f0d516dbf422dafbb1dea14cf405e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:34.914913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=191f0d516dbf422dafbb1dea14cf405e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:34.917005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:34.971520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191f0d516dbf422dafbb1dea14cf405e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:34.973662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:34.981708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191f0d516dbf422dafbb1dea14cf405e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:34.995457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191f0d516dbf422dafbb1dea14cf405e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.003342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191f0d516dbf422dafbb1dea14cf405e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.004102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191f0d516dbf422dafbb1dea14cf405e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.010387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1074/2154] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:34 2022] DELETE /identity/v3/users/191f0d516dbf422dafbb1dea14cf405e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:35.014964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:35.015446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:35.015548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:35.035320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:35.036616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:35.105513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1081/2155] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:35.115041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:35.116509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:35.175307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9198e-1b37-425e-91ba-6b7c2ea106ff admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:35.179056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:35.179681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:35.179965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:35.180240 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:35.190387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:35.192017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:35.202275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1075/2156] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:35.622470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:35.624711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:35.665315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e077c-ea07-4d64-8891-16aefd465393 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:35.677241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:35.677977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:35.678266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:35.678537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] PATH_INFO: `/v3/projects/7b1d877f14b24c63b5c7af23f0568e14` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:35.679862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1d877f14b24c63b5c7af23f0568e14)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:35.681843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:35.735368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:35.745247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1d877f14b24c63b5c7af23f0568e14', 'request_id': 'req-28095e04-6ca1-4297-951d-aceb597af826'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.758948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1d877f14b24c63b5c7af23f0568e14', 'request_id': 'req-28095e04-6ca1-4297-951d-aceb597af826'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.768200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1d877f14b24c63b5c7af23f0568e14', 'request_id': 'req-28095e04-6ca1-4297-951d-aceb597af826'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.782038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] The token cache is being invalidate because project 7b1d877f14b24c63b5c7af23f0568e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:35.782337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.783462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.785523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1082/2157] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:35 2022] DELETE /identity/v3/projects/7b1d877f14b24c63b5c7af23f0568e14 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:35.789713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:35.804948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb3236-ae81-4e86-8127-94dab80dccc8 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:35.873649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:35.874694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:35.875079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:35.875443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] PATH_INFO: `/v3/users/ef40d1456e1b415c9779c251d694ed46` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:35.877785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef40d1456e1b415c9779c251d694ed46)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:35.880410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:35.941823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef40d1456e1b415c9779c251d694ed46'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.950467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef40d1456e1b415c9779c251d694ed46'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.963930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:35.965309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef40d1456e1b415c9779c251d694ed46'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.974272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef40d1456e1b415c9779c251d694ed46'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.975486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ba16d8c-7b9a-4836-bb08-7de1271c6c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef40d1456e1b415c9779c251d694ed46'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:35.982088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1076/2158] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:35 2022] DELETE /identity/v3/users/ef40d1456e1b415c9779c251d694ed46 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:36.493062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:36.494764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:36.508604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28095e04-6ca1-4297-951d-aceb597af826 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:36.577401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:36.578980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:36.579269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:36.579552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] PATH_INFO: `/v3/projects/1c84c5172374433292b2c3e93602c355` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:36.581395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c84c5172374433292b2c3e93602c355)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:36.583850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:36.640372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c84c5172374433292b2c3e93602c355', 'request_id': 'req-1b0340bc-df31-4caa-9059-c430b2f9b3fb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:36.646864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c84c5172374433292b2c3e93602c355', 'request_id': 'req-1b0340bc-df31-4caa-9059-c430b2f9b3fb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:36.653867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c84c5172374433292b2c3e93602c355', 'request_id': 'req-1b0340bc-df31-4caa-9059-c430b2f9b3fb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:36.670315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] The token cache is being invalidate because project 1c84c5172374433292b2c3e93602c355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:36.670883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:36.671524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:36.673898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1083/2159] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:36 2022] DELETE /identity/v3/projects/1c84c5172374433292b2c3e93602c355 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:38.127760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1907d5-dbbe-49ef-afbb-1a89ce37a5d2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.128684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1907d5-dbbe-49ef-afbb-1a89ce37a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.129314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1907d5-dbbe-49ef-afbb-1a89ce37a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.155150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ab1907d5-dbbe-49ef-afbb-1a89ce37a5d2 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:38.157049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ab1907d5-dbbe-49ef-afbb-1a89ce37a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:38.236775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1077/2160] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:10:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:38.251713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:38.254207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:38.331879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:38.361244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0340bc-df31-4caa-9059-c430b2f9b3fb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:38.362531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:38.362531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.362694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.362909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.374022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:38.375978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49466aed-4cf5-448e-a824-785660bfe58d service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:38.386175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1084/2161] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:10:38.641675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.641987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.643352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.645243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1078/2162] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:10:38.691387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.691786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.692164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.717434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:38.722316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:38.779115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:38.781152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:38.815476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1085/2163] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:10:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:38.832111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:38.833736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:38.857281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe14ab-a41d-47c0-a2c8-ab1e44124675 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:38.863765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:38.883921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.884460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.884821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.902531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:38.904576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b596099-c17b-4466-89ac-73dd5f4f708d service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:38.907109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fe631-6294-4742-aac1-19476d7e316c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:38.912643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:38.913324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:38.913603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:38.913879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] PATH_INFO: `/v3/users/ec4599fb9568454ba36b01038c345248` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:38.915545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4599fb9568454ba36b01038c345248)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:38.917206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:38.921401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1086/2164] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:10:38.982904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4599fb9568454ba36b01038c345248'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:38.991526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4599fb9568454ba36b01038c345248'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.013132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4599fb9568454ba36b01038c345248'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.022735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4599fb9568454ba36b01038c345248'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.023451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4599fb9568454ba36b01038c345248'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.031650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1079/2165] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:38 2022] DELETE /identity/v3/users/ec4599fb9568454ba36b01038c345248 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:39.032118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.032416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.032719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.056002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:39.057590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.135402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1087/2166] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:39.143448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.144894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.162027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:39.237827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0212f-da04-4293-914f-12dac4057af7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.242977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.244752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.256095 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.256851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.257108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.257390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.269610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.271256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9287fb-d8b9-40fc-9c92-297cf653c7ee service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.280182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1080/2167] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:10:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:10:39.355004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde464b-60d2-4d94-80fd-c72495a35ff6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.365756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.366660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.367137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.367519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] PATH_INFO: `/v3/projects/6cab65810db34765966b2ca0332088be` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.368914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cab65810db34765966b2ca0332088be)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.370945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.390374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:39.432561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cab65810db34765966b2ca0332088be', 'request_id': 'req-171857ac-e8bb-47bf-86b4-dac8621f7755'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.438593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cab65810db34765966b2ca0332088be', 'request_id': 'req-171857ac-e8bb-47bf-86b4-dac8621f7755'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.445493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cab65810db34765966b2ca0332088be', 'request_id': 'req-171857ac-e8bb-47bf-86b4-dac8621f7755'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.460853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] The token cache is being invalidate because project 6cab65810db34765966b2ca0332088be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:39.461179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.461812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-171857ac-e8bb-47bf-86b4-dac8621f7755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:39.463333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1088/2168] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:39 2022] DELETE /identity/v3/projects/6cab65810db34765966b2ca0332088be => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:39.469463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b988-d613-4e4f-8956-b9eb4a2ef0a5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.469852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b988-d613-4e4f-8956-b9eb4a2ef0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.470757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b988-d613-4e4f-8956-b9eb4a2ef0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.492228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f697b988-d613-4e4f-8956-b9eb4a2ef0a5 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:39.493482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f697b988-d613-4e4f-8956-b9eb4a2ef0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.537745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.537916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.538158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.558096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1081/2169] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:39.565551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:39.566922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.567367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.567960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.568287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.589989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:39.591216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.600692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1082/2170] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:39.605987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.617777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd65cc4-edf0-4649-aa62-e6b53e80e8ce None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.624545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.625317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.625788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.626071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.627568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:10:39.628225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.631372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.638301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1089/2171] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:39.651475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1083/2172] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:10:39.651815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.653083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:39.653865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.667346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1caf4-1ab4-4135-8e72-83aefdb0fcd1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.672854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.673765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.674530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.675054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.676143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.678701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.719074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1084/2173] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:39.720092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.731648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf8da0-5d9d-4359-afcf-2e06b4d9b217 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.779406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:39.788380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b83a-f0ba-4f31-987f-48a1de2b5e02 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.793425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.794041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.794325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.794602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.795261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.795826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.796446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.796733 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.797044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.797469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.806487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:39.825412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1085/2174] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:10:39.826428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.838716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bf8e9-bb2c-4500-9da4-77756046b39e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.843632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.844273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.844556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.844832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.846421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.848348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.861146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1086/2175] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:39.862615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.864436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.865142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.873478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1090/2176] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:10:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 20:10:39.879917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75994-4cc7-4b8c-95fb-ffeded3e5409 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.885297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.885934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.886215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.886494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.887192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.888804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90232a4-600b-4c2d-af59-4fe61b7bfd5c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.901589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1087/2177] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:39.905943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:39.918544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c6de-2bfb-42ee-854f-72deb4b3ac05 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:39.924640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:39.925563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.925922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.926380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] PATH_INFO: `/v3/projects/f638f26bc86e4c8893325e8f8d468286/users/fcf3aca3dc0242f6b5f3758c09a7b27f/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:39.953414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f638f26bc86e4c8893325e8f8d468286, user_id=fcf3aca3dc0242f6b5f3758c09a7b27f, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:39.956833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:39.979679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1091/2178] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:39 2022] PUT /identity/v3/projects/f638f26bc86e4c8893325e8f8d468286/users/fcf3aca3dc0242f6b5f3758c09a7b27f/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:39.992673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f194-34ed-4d22-a09d-34034ad99ae8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:39.993118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f194-34ed-4d22-a09d-34034ad99ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:39.993472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f194-34ed-4d22-a09d-34034ad99ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:40.037755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7e63f194-34ed-4d22-a09d-34034ad99ae8 None None] MFA Rules not processed for user `fcf3aca3dc0242f6b5f3758c09a7b27f`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:40.038996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7e63f194-34ed-4d22-a09d-34034ad99ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:40.129927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1088/2179] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:40.140077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:40.141942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:40.205337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f1a2d-4294-4b15-9d0f-0083a8c6233d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:40.208984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:40.209594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:40.209874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:40.210154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:40.218681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:40.220104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:40.229044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1092/2180] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:40.298865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:40.299818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:40.300107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:40.320588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:40.321927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:40.381996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1089/2181] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:10:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:40.396309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:40.467183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16214135-22f1-4002-9f4b-ee3a8b487401 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:40.472650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:40.473324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:40.473604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:40.473930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:40.493278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:40.495644 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a355186-e35f-4c29-a672-bcf025ee6c7d service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:40.505956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1093/2182] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:10:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:10:42.656205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:42.658310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:42.721882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de5d96-4a7a-487a-8739-fa16ce354d79 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:42.726366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:42.726995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:42.727273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:42.727542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] PATH_INFO: `/v3/users/f6aba59bcb664b8bbc1469d71b6a276d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:42.728686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6aba59bcb664b8bbc1469d71b6a276d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:42.730711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:42.733137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:42.778145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6aba59bcb664b8bbc1469d71b6a276d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:42.785223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6aba59bcb664b8bbc1469d71b6a276d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:42.797668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6aba59bcb664b8bbc1469d71b6a276d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:42.804604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6aba59bcb664b8bbc1469d71b6a276d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:42.805292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6aba59bcb664b8bbc1469d71b6a276d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:42.811103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1090/2183] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:42 2022] DELETE /identity/v3/users/f6aba59bcb664b8bbc1469d71b6a276d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:42.815602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:42.816136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:42.816136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:42.823582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:42.837362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:42.838553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:42.897270 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1094/2184] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:42.906668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:42.908693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:42.967206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520373bd-9c48-4fe1-a664-b9cd1cf40855 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:42.971477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:42.972089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:42.972373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:42.972643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:42.981827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:42.983256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0dfae-cff6-4aa2-b19f-70411b5a574d service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:42.992892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1091/2185] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:43.247188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.248983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:43.268265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea43dd6-0517-40dd-897b-0c8f243323f2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.276661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.277391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.277753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.277972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] PATH_INFO: `/v3/projects/725810398afd433c9bff3fcd261b0f6f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.279255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=725810398afd433c9bff3fcd261b0f6f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.281919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.351209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725810398afd433c9bff3fcd261b0f6f', 'request_id': 'req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:43.358896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725810398afd433c9bff3fcd261b0f6f', 'request_id': 'req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:43.366191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725810398afd433c9bff3fcd261b0f6f', 'request_id': 'req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:43.394694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] The token cache is being invalidate because project 725810398afd433c9bff3fcd261b0f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:43.399861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:43.405948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-597cff72-f8e0-4dd1-8ec1-898bfa20e2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:43.407989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1095/2186] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:43 2022] DELETE /identity/v3/projects/725810398afd433c9bff3fcd261b0f6f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:43.413505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.413875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.414242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.435435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:43.436731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:43.496379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1092/2187] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:43.505778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.506134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.506504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.526375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:43.527585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:43.535940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1096/2188] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:43.540207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.550528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4314f67-efc4-404f-bf10-a3bc89cca6bb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.555207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.555829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.556112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.556387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.557159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:10:43.557464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.559064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.571871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1093/2189] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:10:43.576146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.586702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35410a2-34cf-403e-a429-a2e9e660ca9e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.591444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.592091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.592354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.592634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.593330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.595452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.625427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1097/2190] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:43 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:43.630327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.640221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e5d90-ab23-4f7a-8c84-79741de02c0a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.649197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:43.704939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.705769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.706173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.706460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.707144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.709680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.731325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1094/2191] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:43 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:43.735643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.748389 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b23be-6957-4a8e-acf9-0c5ed6c7b59f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.754012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.754625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.754902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.755176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.756035 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.757724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.768215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1098/2192] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:43.772298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.782176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326c744-9bc7-41fb-96c0-1730de0b9544 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.787470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.788068 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.788346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.788616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.789329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.790908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7695f-70f6-4df8-8f6e-49796bafe058 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.801961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1095/2193] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:43.806066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.816741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ac8bc-50e2-4fee-8cfa-740755bab43b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:43.821769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:43.822426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.822707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.822986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] PATH_INFO: `/v3/projects/87d5b073df6b48908d97ae613731adc1/users/5f8ecfcb241d468685b22c3f883b6623/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.838718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d5b073df6b48908d97ae613731adc1, user_id=5f8ecfcb241d468685b22c3f883b6623, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:43.841462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:43.855252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1099/2194] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:43 2022] PUT /identity/v3/projects/87d5b073df6b48908d97ae613731adc1/users/5f8ecfcb241d468685b22c3f883b6623/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:43.860304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6c0d-97a6-4ce3-8d2c-406a37cb4e68 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:43.860589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6c0d-97a6-4ce3-8d2c-406a37cb4e68 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:43.861463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6c0d-97a6-4ce3-8d2c-406a37cb4e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:43.881665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-990d6c0d-97a6-4ce3-8d2c-406a37cb4e68 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:43.883121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-990d6c0d-97a6-4ce3-8d2c-406a37cb4e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:43.943343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1096/2195] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:43.952789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:43.954494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:43.980778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:44.012777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bd6f-d27e-4dea-b80a-64eafdf8f308 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:44.016457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:44.017135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:44.017416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:44.017691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:44.026556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:44.027988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:44.038329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1100/2196] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:48.981150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:48.981607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:48.981908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.025886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] MFA Rules not processed for user `5f8ecfcb241d468685b22c3f883b6623`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:49.027011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:49.110754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1097/2197] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:49.117066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.127827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260e4a4-8987-4eec-9b63-e732a0a27d3a service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.132713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.133452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.133796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.134283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.135089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.137186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.170306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1101/2198] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:49 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:49.174885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.186235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cfdf8-f4ce-4d42-9abf-37adb9a158ad None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.208335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:49.285300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.286040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.286328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.286607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.287301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.289539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.312848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1098/2199] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:49 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:10:49.317139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.328286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e9913f-c80a-419c-96e7-5c018b6e8b33 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.333378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.334007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.334290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.334566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.335266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.336891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.350997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1102/2200] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:49.353497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.365233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a21586-74b6-4b8c-a114-d06392765558 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.370125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.370812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.371091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.371396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] PATH_INFO: `/v3/projects/6f94b946276c470b8483c8d909cafacf/users/6bd8078f37294bb29879b3b57f4c3466/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.388223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f94b946276c470b8483c8d909cafacf, user_id=6bd8078f37294bb29879b3b57f4c3466, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.390967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.405768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1099/2201] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:49 2022] PUT /identity/v3/projects/6f94b946276c470b8483c8d909cafacf/users/6bd8078f37294bb29879b3b57f4c3466/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:49.410404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.420905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1768be-60b9-4b5a-ba51-c23986423dbd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.483055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.483469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.483615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.483893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.484631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.486391 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.497986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1103/2202] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:49.502142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.514393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.516168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:49.517775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687c201-7fb5-467c-a6f8-aa751f7abedc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.522941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.523752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.524105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.524459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] PATH_INFO: `/v3/projects/6f94b946276c470b8483c8d909cafacf/users/6bd8078f37294bb29879b3b57f4c3466/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.526662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f94b946276c470b8483c8d909cafacf, user_id=6bd8078f37294bb29879b3b57f4c3466, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.529431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e597b89-515e-4a58-ad8d-7aada35cb494 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.529583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99899a76-190f-435f-881f-02cfaa930eb7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.535191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.535922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.536295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.536759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] PATH_INFO: `/v3/users/fcf3aca3dc0242f6b5f3758c09a7b27f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.538180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf3aca3dc0242f6b5f3758c09a7b27f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.540212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.542006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1100/2203] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:49 2022] PUT /identity/v3/projects/6f94b946276c470b8483c8d909cafacf/users/6bd8078f37294bb29879b3b57f4c3466/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:49.599347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf3aca3dc0242f6b5f3758c09a7b27f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:49.614471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf3aca3dc0242f6b5f3758c09a7b27f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:49.615464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf3aca3dc0242f6b5f3758c09a7b27f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:49.625995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf3aca3dc0242f6b5f3758c09a7b27f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:49.636873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf3aca3dc0242f6b5f3758c09a7b27f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:49.639989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:49.645753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1104/2204] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:10:49 2022] DELETE /identity/v3/users/fcf3aca3dc0242f6b5f3758c09a7b27f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:49.650415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.650664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.651278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.674720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:49.676371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:49.789145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1101/2205] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:49.796878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:49.803831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:49.896615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4bf2f-97a8-40dd-9a22-6bb290579533 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:49.899893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:49.900534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:49.900815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:49.901189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:49.915096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:49.916618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42a015-6286-4f19-b52b-eb407db1190f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:49.929811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1105/2206] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:50.609385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:50.612670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:50.628421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbeef2-adbe-4e29-8a72-b61c07fd305f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:50.633633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:50.634413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:50.634758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:50.635104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] PATH_INFO: `/v3/projects/f638f26bc86e4c8893325e8f8d468286` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:50.636355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f638f26bc86e4c8893325e8f8d468286)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:50.638596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:50.730614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f638f26bc86e4c8893325e8f8d468286', 'request_id': 'req-ca037d17-6ff1-4377-a24a-0f5c448eed69'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:50.738180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f638f26bc86e4c8893325e8f8d468286', 'request_id': 'req-ca037d17-6ff1-4377-a24a-0f5c448eed69'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:50.745292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f638f26bc86e4c8893325e8f8d468286', 'request_id': 'req-ca037d17-6ff1-4377-a24a-0f5c448eed69'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:50.750236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:50.765416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] The token cache is being invalidate because project f638f26bc86e4c8893325e8f8d468286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:10:50.766259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:50.767520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ca037d17-6ff1-4377-a24a-0f5c448eed69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:10:50.769617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1102/2207] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:10:50 2022] DELETE /identity/v3/projects/f638f26bc86e4c8893325e8f8d468286 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:50.774212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:50.775052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:50.775399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:50.796518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:50.797953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:50.879784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1106/2208] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:10:50.887754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:50.888149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:50.888434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:50.924922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:50.926402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:50.946533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1103/2209] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:50.951242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:50.964717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dca2ae-5883-4577-89fa-568a4bb03533 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:50.972454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:50.973472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:50.981324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:50.981942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:50.982891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:10:50.983355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:50.985480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.002415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1107/2210] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:10:51.006143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.015787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd39f2-904b-403c-8b4a-cc2dffe610aa None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.024549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.025311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.025684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.026565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.026785 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.028782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.072321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1104/2211] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:51 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:51.077742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.089619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbece5-44ca-476b-8b11-36edd8234ea5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.164731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.165490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.165846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.166197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.167176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.169346 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.199794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1108/2212] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:51 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:51.205000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.215826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fa16b-b4fe-4f9e-b16e-488a9da58afc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.222234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.222913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.223142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.223421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.224527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.225918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.242025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.245120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1105/2213] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:51.253781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f6ba5-1560-47cc-97fe-b5aec072ac48 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.259940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.260575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.260858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.261186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.261890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.263507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caff08d-674a-47d1-85bb-b36a9ef902f0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.278599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1109/2214] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:51.283433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.300988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1372c-bf10-4e6c-ab66-9e4b7daf8a9b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.307612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.308890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.308890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.309134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] PATH_INFO: `/v3/projects/e903fc5c44264331a5f4ff2d882321f4/users/f2bfc695b7e8460597f705da0b1f2cd9/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.328442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e903fc5c44264331a5f4ff2d882321f4, user_id=f2bfc695b7e8460597f705da0b1f2cd9, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.332617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.368910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1106/2215] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:51 2022] PUT /identity/v3/projects/e903fc5c44264331a5f4ff2d882321f4/users/f2bfc695b7e8460597f705da0b1f2cd9/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:10:51.372837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.373659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.374086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.399744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:51.416604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] MFA Rules not processed for user `f2bfc695b7e8460597f705da0b1f2cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:51.417950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:51.514625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1110/2216] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:51.520127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.626671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad4dc2-3aea-44ac-a0ad-b078d527ce80 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.637715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.638667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.639108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.639540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.640674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.650732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.698147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1107/2217] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:10:51 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:51.701723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.717257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549dc10-c9dd-4481-865c-710c398470c3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.791308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.792105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.792491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.792870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.793970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.796059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.867374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1111/2218] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:10:51 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:10:51.872094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.898646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668180-07b8-42f9-8961-9fb15527e394 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.907466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.908790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.909281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.909696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.910692 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.913708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:51.936352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1108/2219] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:10:51.945874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:51.957424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25841eea-6c21-4aa4-bf41-6a971fdc0e53 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:51.962561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:51.963379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:51.963777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:51.964179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] PATH_INFO: `/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9/users/25a4ca3a1b7548c6818ebb797022f833/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:51.987121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebe3f01786849f4ac0711a0eef8c2f9, user_id=25a4ca3a1b7548c6818ebb797022f833, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:51.990732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:52.010515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1112/2220] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:51 2022] PUT /identity/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9/users/25a4ca3a1b7548c6818ebb797022f833/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:52.015865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:52.028309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821fcc7-976f-4cf8-a9aa-5d1f6fdde010 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:52.132995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:52.133305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:52.133588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:52.133859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:52.134549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:52.136158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0af5b9-5b7b-4c5d-bbb6-3e9dd85dbb10 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:52.147559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1109/2221] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:10:52.151502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:52.165521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8821b-9f34-4431-8554-c28bd6d2167a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:52.176268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:52.177323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:52.177808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:52.178189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] PATH_INFO: `/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9/users/25a4ca3a1b7548c6818ebb797022f833/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:52.180484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebe3f01786849f4ac0711a0eef8c2f9, user_id=25a4ca3a1b7548c6818ebb797022f833, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:52.183619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:52.224419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1113/2222] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:10:52 2022] PUT /identity/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9/users/25a4ca3a1b7548c6818ebb797022f833/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:10:52.231031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:52.231754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:52.232205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:52.244119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:52.275925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] MFA Rules not processed for user `25a4ca3a1b7548c6818ebb797022f833`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:52.279144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:52.359926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1110/2223] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:52.381353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:52.384257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:52.503239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9808783-f7be-4b83-959d-c704ceb32af8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:52.513378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:52.514044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:52.514404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:52.514682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:52.527550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:52.535112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:52.537242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708f06c-64f2-46c0-b32b-f2188f4f40f0 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:52.564117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1114/2224] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:52.669189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:52.681178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee2b73-203b-46d0-96dc-6cee916e24d5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:52.689295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:52.690360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:52.690649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:52.690928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:52.829710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:52.832274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:52.852994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1111/2225] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:10:55.487349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21262aa-25c7-4c57-90b4-9bf84736e2c9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:55.487861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21262aa-25c7-4c57-90b4-9bf84736e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:55.488336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21262aa-25c7-4c57-90b4-9bf84736e2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:55.516413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a21262aa-25c7-4c57-90b4-9bf84736e2c9 None None] MFA Rules not processed for user `6bd8078f37294bb29879b3b57f4c3466`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:55.517646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a21262aa-25c7-4c57-90b4-9bf84736e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:55.592271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1115/2226] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:10:55.608126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:10:55.609579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:55.687978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ef9b8-2dc7-4e2d-8408-f587bab6e651 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:10:55.694563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:10:55.695205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:55.695488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:55.695763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:55.697576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:55.719487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:10:55.758403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:10:55.760060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:10:55.769421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1112/2227] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:10:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:10:56.970850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:10:56.970850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:10:56.971438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:10:56.992390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] MFA Rules not processed for user `5f8ecfcb241d468685b22c3f883b6623`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:10:56.993631 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:10:57.007073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1116/2228] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:03.240778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:03.242347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:03.300466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a158-40c4-421b-b47e-8c5ac9aefeab service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:03.304033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:03.304649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:03.304931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:03.305325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:03.307091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:03.371265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:03.372868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182d6e4-efe0-429a-81b1-0a16765e7250 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:03.383502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1113/2229] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:04.449614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:04.537922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc5271-b3c8-4dd3-86f1-78aaa2f6d759 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:04.542607 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:04.543267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:04.543553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:04.543833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] PATH_INFO: `/v3/users/f2bfc695b7e8460597f705da0b1f2cd9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:04.545225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bfc695b7e8460597f705da0b1f2cd9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:04.547186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:04.611530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bfc695b7e8460597f705da0b1f2cd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:04.626716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bfc695b7e8460597f705da0b1f2cd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:04.627872 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bfc695b7e8460597f705da0b1f2cd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:04.636148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bfc695b7e8460597f705da0b1f2cd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:04.638212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:04.647392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bfc695b7e8460597f705da0b1f2cd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:04.659018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1117/2230] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:11:04 2022] DELETE /identity/v3/users/f2bfc695b7e8460597f705da0b1f2cd9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:04.660356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:04.660836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:04.660924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:04.683808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:04.685187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:04.770432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1114/2231] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:04.781640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:04.784500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:04.892988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ec6ec-3bd2-4462-9c6e-441c8386f7be admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:04.897484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:04.898216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:04.898569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:04.898914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:04.916888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:04.918374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656962a7-387b-43d4-a80b-13a2c6277d13 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:04.930352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1118/2232] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:11:05.569509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:05.572050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:05.591594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d7efd-3a59-4573-89fd-1fcce9ec78f8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:05.602551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:05.612356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:05.614467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:05.615177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] PATH_INFO: `/v3/projects/e903fc5c44264331a5f4ff2d882321f4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:05.618125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e903fc5c44264331a5f4ff2d882321f4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:05.621216 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:05.632145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:05.633019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:05.633537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:05.633639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:05.668011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] MFA Rules not processed for user `a48eaef4d70b4116bd307374d819bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:05.678353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:05.695436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e903fc5c44264331a5f4ff2d882321f4', 'request_id': 'req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:05.703422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e903fc5c44264331a5f4ff2d882321f4', 'request_id': 'req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:05.713574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e903fc5c44264331a5f4ff2d882321f4', 'request_id': 'req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:05.742550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] The token cache is being invalidate because project e903fc5c44264331a5f4ff2d882321f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:11:05.743303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:05.745235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:05.750020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1115/2233] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:11:05 2022] DELETE /identity/v3/projects/e903fc5c44264331a5f4ff2d882321f4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:05.754421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:05.756250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:05.783212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1119/2234] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:05.879858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb65c-b045-4433-8adb-f42a10aaa1b5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:05.886107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:05.887114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:05.887471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:05.887843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] PATH_INFO: `/v3/users/25a4ca3a1b7548c6818ebb797022f833` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:05.889326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a4ca3a1b7548c6818ebb797022f833)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:05.900121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:06.014940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a4ca3a1b7548c6818ebb797022f833'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:06.026995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a4ca3a1b7548c6818ebb797022f833'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:06.054688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a4ca3a1b7548c6818ebb797022f833'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:06.066447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a4ca3a1b7548c6818ebb797022f833'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:06.067233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b872abdc-0c90-45a5-b97b-0fd277e3dc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a4ca3a1b7548c6818ebb797022f833'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:06.076323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1116/2235] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:11:05 2022] DELETE /identity/v3/users/25a4ca3a1b7548c6818ebb797022f833 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:06.764590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:06.806892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9795-94cb-4a8b-80a3-34c570df62e6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:06.931629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:06.932489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:06.932840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:06.933254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] PATH_INFO: `/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:06.934655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebe3f01786849f4ac0711a0eef8c2f9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:06.938623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:06.999539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:07.013708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebe3f01786849f4ac0711a0eef8c2f9', 'request_id': 'req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:07.021453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebe3f01786849f4ac0711a0eef8c2f9', 'request_id': 'req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:07.028396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebe3f01786849f4ac0711a0eef8c2f9', 'request_id': 'req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:07.046655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] The token cache is being invalidate because project cebe3f01786849f4ac0711a0eef8c2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:11:07.046935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:07.048428 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ebf54b6-0bb6-47c0-ab2c-a90a124d3135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:07.057454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1120/2236] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:11:06 2022] DELETE /identity/v3/projects/cebe3f01786849f4ac0711a0eef8c2f9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:07.059242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.059746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.059857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.088648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:07.092865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:07.112675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:07.188631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1117/2237] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:07.197434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.198021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.198827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.232672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:07.234635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:07.247525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1121/2238] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:07.252599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.264243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d25b3-17d3-4fd9-a580-35324bba1cc6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.269558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.270192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.271174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.271534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.272308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:11:07.272678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.275116 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.291547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1118/2239] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:11:07.296327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.315890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d43e8-9244-4bba-a134-ec7dfa83e110 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.322903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.323535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.323820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.324098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.324906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.327023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.361347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1122/2240] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:11:07.367691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.394730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e028-a26a-4d6b-bacd-555ebc0e6035 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.478854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.479592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.479949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.480295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.481066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.483078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.511505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1119/2241] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:11:07.515699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.540248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c6af1-4a85-4f88-9a2c-9801a15513e6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.552259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.553111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.553512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.553888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.554749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.556868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.569545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1123/2242] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:07.573227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.585752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bdad9-6f07-4c2d-ab79-531cfec627b5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.592400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.593417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.593828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.595021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.596430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.598359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c232413-73ef-4f5e-b8b3-1446c447b140 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.611784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1120/2243] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:11:07.618117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.649748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5137d3-dc42-46b6-9248-d40ff8e12512 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.657982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.658711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.659000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.659285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] PATH_INFO: `/v3/projects/11f169c45fdd468483ba06e61c411f7f/users/d62c5b877df841b1be720d6a542c17ad/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.680587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f169c45fdd468483ba06e61c411f7f, user_id=d62c5b877df841b1be720d6a542c17ad, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.691713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.710233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1124/2244] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:07 2022] PUT /identity/v3/projects/11f169c45fdd468483ba06e61c411f7f/users/d62c5b877df841b1be720d6a542c17ad/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:07.722234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.722745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.723175 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.761375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] MFA Rules not processed for user `d62c5b877df841b1be720d6a542c17ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:07.762609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:07.837045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1121/2245] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:11:07.841421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.925602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8562b-c64b-4b27-a1c9-e0bbf87a5709 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.930481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:07.931287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:07.931612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:07.932361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:07.932752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:07.935060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:07.965116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1125/2246] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:11:07.969468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:07.983451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10644f10-62d6-4994-8e01-ae80a649369a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:07.991970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:08.049916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.050545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.050823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.051096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.051765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.053732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.082956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1122/2247] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:11:07 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:11:08.084602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:08.110978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87361f8f-2615-4c28-bc71-2e9839031849 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:08.116632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.117452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.117861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.118353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.119129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.120859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.133462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1126/2248] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:08.137933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:08.154569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35918cb-8b15-4260-ad79-7e972f4b2205 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:08.160754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.161957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.162300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.162937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] PATH_INFO: `/v3/projects/50800d0cc0194603acdc4d8acd4819a4/users/86f5d8ec49804bba858e936eeb07728d/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.214003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=50800d0cc0194603acdc4d8acd4819a4, user_id=86f5d8ec49804bba858e936eeb07728d, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.219790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.254649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1123/2249] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:08 2022] PUT /identity/v3/projects/50800d0cc0194603acdc4d8acd4819a4/users/86f5d8ec49804bba858e936eeb07728d/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:11:08.254974 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:08.268647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fc2f6-0ac9-4854-b0f8-4d31048cf2e7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:08.394466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:08.429649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.430548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.430951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.431231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.432342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.434184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2311960-7caa-4540-8dc0-ef1aa51f2f85 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.451204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1127/2250] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:11:08.452230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:08.470553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eef1aa-a46a-4334-bb6d-8ec7e1f3f5ed admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:08.477703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.478409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.478824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.479114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] PATH_INFO: `/v3/projects/50800d0cc0194603acdc4d8acd4819a4/users/86f5d8ec49804bba858e936eeb07728d/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.483187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=50800d0cc0194603acdc4d8acd4819a4, user_id=86f5d8ec49804bba858e936eeb07728d, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.487115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.518950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1124/2251] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:08 2022] PUT /identity/v3/projects/50800d0cc0194603acdc4d8acd4819a4/users/86f5d8ec49804bba858e936eeb07728d/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:08.529408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.529696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.530394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.563123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] MFA Rules not processed for user `86f5d8ec49804bba858e936eeb07728d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:08.564287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:08.675810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1128/2252] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:08.682007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:08.683993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:08.772769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d6-1a6c-4969-837a-cf2359c867ee admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:08.778462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:08.779176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:08.779465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:08.779879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:08.871589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:08.873030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:08.909716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1125/2253] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:16.833228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:16.842726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37486e2-f7c1-469b-bad1-884e69063153 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:16.847286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:16.847831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:16.848109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:16.848385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:16.857825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:16.859257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:16.869663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1129/2254] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:18.880380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:26.865274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=51 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:31.298769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:31.300687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:31.393450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82947c40-548f-4071-ac4a-b34d02b33b9f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:31.399612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:31.400617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:31.400933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:31.401478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:31.403713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:31.421751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:31.423427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a840c14-cf55-48a1-92e3-f5c358da7c39 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:31.436255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1126/2255] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:41.431379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=32 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:52.216553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:52.279374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4312b2-fc46-4ec8-a607-4bc3b5eda1b9 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:52.284007 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:52.284634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:52.285205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:52.285480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] PATH_INFO: `/v3/users/d62c5b877df841b1be720d6a542c17ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:52.286621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62c5b877df841b1be720d6a542c17ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:52.288568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:52.333233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62c5b877df841b1be720d6a542c17ad'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.344781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62c5b877df841b1be720d6a542c17ad'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.345503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62c5b877df841b1be720d6a542c17ad'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.352225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62c5b877df841b1be720d6a542c17ad'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.358675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62c5b877df841b1be720d6a542c17ad'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.364683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1130/2256] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:11:52 2022] DELETE /identity/v3/users/d62c5b877df841b1be720d6a542c17ad => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:52.369308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:52.369767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:52.369875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:52.389282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:52.391074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:52.448560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1127/2257] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:52.458096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:52.459493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:52.515639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18502a-f384-4111-a5f2-3d2306f100c0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:52.519251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:52.519851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:52.520155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:52.520426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:52.529374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:52.530961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:52.540180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1131/2258] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:52.812540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:52.814030 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:52.826722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71380838-0321-45c5-8f05-dfcd708e3f07 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:52.832718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:52.833851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:52.834224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:52.834601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] PATH_INFO: `/v3/projects/11f169c45fdd468483ba06e61c411f7f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:52.836162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f169c45fdd468483ba06e61c411f7f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:52.838938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:52.891989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f169c45fdd468483ba06e61c411f7f', 'request_id': 'req-056d3e76-f4e0-4563-a144-77741d653544'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.897451 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f169c45fdd468483ba06e61c411f7f', 'request_id': 'req-056d3e76-f4e0-4563-a144-77741d653544'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.903582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f169c45fdd468483ba06e61c411f7f', 'request_id': 'req-056d3e76-f4e0-4563-a144-77741d653544'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.915991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] The token cache is being invalidate because project 11f169c45fdd468483ba06e61c411f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:11:52.916275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.916901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:52.918444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1128/2259] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:11:52 2022] DELETE /identity/v3/projects/11f169c45fdd468483ba06e61c411f7f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:52.923137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:52.932605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858b80-0bd1-492a-b1d4-b682fc2dd819 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:52.950286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:52.990100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:52.990752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:52.991036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:52.991316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] PATH_INFO: `/v3/users/86f5d8ec49804bba858e936eeb07728d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:52.992800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f5d8ec49804bba858e936eeb07728d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:52.994783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.044518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f5d8ec49804bba858e936eeb07728d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.056771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f5d8ec49804bba858e936eeb07728d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.057477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f5d8ec49804bba858e936eeb07728d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.064902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f5d8ec49804bba858e936eeb07728d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.074204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd396af8-2ff5-4760-898f-56e78c9ab6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f5d8ec49804bba858e936eeb07728d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.085425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1132/2260] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:11:52 2022] DELETE /identity/v3/users/86f5d8ec49804bba858e936eeb07728d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:53.364943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.366431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:53.380082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d3e76-f4e0-4563-a144-77741d653544 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.436307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.437602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.437890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.438167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] PATH_INFO: `/v3/projects/50800d0cc0194603acdc4d8acd4819a4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.439333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50800d0cc0194603acdc4d8acd4819a4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.441301 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.471664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:53.489659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50800d0cc0194603acdc4d8acd4819a4', 'request_id': 'req-565e2c20-dc49-4858-be86-b5202d7a16f4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.495215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50800d0cc0194603acdc4d8acd4819a4', 'request_id': 'req-565e2c20-dc49-4858-be86-b5202d7a16f4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.502027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50800d0cc0194603acdc4d8acd4819a4', 'request_id': 'req-565e2c20-dc49-4858-be86-b5202d7a16f4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.514122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] The token cache is being invalidate because project 50800d0cc0194603acdc4d8acd4819a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:11:53.514414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.515197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-565e2c20-dc49-4858-be86-b5202d7a16f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:11:53.516738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1129/2261] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:11:53 2022] DELETE /identity/v3/projects/50800d0cc0194603acdc4d8acd4819a4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:53.521679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337732fd-453f-434d-be16-22698340fa0b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.522140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337732fd-453f-434d-be16-22698340fa0b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.522242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337732fd-453f-434d-be16-22698340fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.541429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-337732fd-453f-434d-be16-22698340fa0b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:53.542623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-337732fd-453f-434d-be16-22698340fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:53.601493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1133/2262] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:53.611087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.611377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.611659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.631213 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:53.632425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:53.640686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1130/2263] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:53.644878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337732fd-453f-434d-be16-22698340fa0b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.654766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337732fd-453f-434d-be16-22698340fa0b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.659804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.660409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.660719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.661024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.661722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:11:53.662030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.663632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.675895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1134/2264] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:11:53.680062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.689308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dcceb-5712-4c93-a710-67bd238966e8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.694248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.694901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.695187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.695461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.696134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.698090 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.725929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1131/2265] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:11:53 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:11:53.729841 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.740678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73623e7a-606f-4de8-ba87-631e7d433721 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.767863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:53.800058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.800684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.801017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.801287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.801965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.803894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.825190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1135/2266] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:11:53 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:11:53.829957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.839991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85d629-9bfd-41cb-83cb-7323876f15f0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.844475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.845115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.845397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.845669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.846355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.847931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.857554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1132/2267] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:53.861487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.871213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90757898-b440-46a1-b5f8-e8b0bd679ba6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.876584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.877222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.877500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.877773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.878458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.880258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3258832-d99f-49e5-b561-92a997ed0e59 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.892273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1136/2268] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:53.896408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:53.906660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ae1b-9beb-41cb-98d1-23b6ec5c96af admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:53.911327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:53.911992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.912277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.912552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] PATH_INFO: `/v3/projects/494bac11d92540c1aa4a72da53f2d265/users/f49c48be0bf74bc59165657b8cbf7de8/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.928277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494bac11d92540c1aa4a72da53f2d265, user_id=f49c48be0bf74bc59165657b8cbf7de8, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:53.930975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:53.944631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1133/2269] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:53 2022] PUT /identity/v3/projects/494bac11d92540c1aa4a72da53f2d265/users/f49c48be0bf74bc59165657b8cbf7de8/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:53.949880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda37a-c1c2-4f53-b443-20902cc6e07a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:53.950177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda37a-c1c2-4f53-b443-20902cc6e07a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:53.950458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fda37a-c1c2-4f53-b443-20902cc6e07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:53.969719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-21fda37a-c1c2-4f53-b443-20902cc6e07a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:53.970918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21fda37a-c1c2-4f53-b443-20902cc6e07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:54.030735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1137/2270] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:54.040587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:54.042642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:54.100921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5aaa-b229-4f24-ab51-cd4b04e68ab7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:54.104648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:54.105415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:54.105721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:54.105995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:54.115798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:54.117244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:54.127329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1134/2271] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:11:54.764009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:54.764779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:54.764779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:54.793075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] MFA Rules not processed for user `f49c48be0bf74bc59165657b8cbf7de8`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:54.794291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:54.861903 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1138/2272] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:11:54.866469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:54.877244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2d516-8cbd-45b1-b04f-5f82236b24ad service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:54.883395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:54.884021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:54.884302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:54.884579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:54.885289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:54.887224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:54.915337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1135/2273] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:11:54 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:11:54.919666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:54.931017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3532d71-bcab-4f8c-bb42-165272a7a4cb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:54.969528 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:54.994552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:54.995173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:54.995451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:54.995723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:54.996442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:54.998375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:55.021944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1139/2274] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:11:54 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:11:55.025929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:55.036328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016b25a-8491-47d5-8a56-821a2696520a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:55.040990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:55.041599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:55.041885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:55.042176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:55.042871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:55.044680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:55.055589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1136/2275] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:55.059574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:55.070531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3298af8-77a2-4b86-9d5e-3f18197ffbbb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:55.075651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:55.076488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:55.076659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:55.077125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] PATH_INFO: `/v3/projects/7f01f53583bb4edc917aa64a50746996/users/1b42f5aee6e14f51a669a375d7dd1b31/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:55.092928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f01f53583bb4edc917aa64a50746996, user_id=1b42f5aee6e14f51a669a375d7dd1b31, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:55.095701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:55.108708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1140/2276] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:55 2022] PUT /identity/v3/projects/7f01f53583bb4edc917aa64a50746996/users/1b42f5aee6e14f51a669a375d7dd1b31/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:11:55.112571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:55.122428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69ea1-3d52-4225-9fbf-034802d56015 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:55.138435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:11:55.181550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:55.182182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:55.182449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:55.182727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:55.183417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:55.185065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9362b6-61eb-44b8-ae1e-e9c1bdcde0b4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:55.195134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1137/2277] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:11:55.199214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:55.209301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4533f6-a421-49aa-a4eb-28decaff5b6d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:55.213740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:55.214385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:55.214663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:55.214936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] PATH_INFO: `/v3/projects/7f01f53583bb4edc917aa64a50746996/users/1b42f5aee6e14f51a669a375d7dd1b31/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:55.216847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f01f53583bb4edc917aa64a50746996, user_id=1b42f5aee6e14f51a669a375d7dd1b31, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:55.220004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:55.230041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1141/2278] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:11:55 2022] PUT /identity/v3/projects/7f01f53583bb4edc917aa64a50746996/users/1b42f5aee6e14f51a669a375d7dd1b31/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:11:55.928332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:55.928867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:55.928867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:55.959253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] MFA Rules not processed for user `1b42f5aee6e14f51a669a375d7dd1b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:11:55.960499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:56.033098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1138/2279] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:11:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:11:56.047176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:56.049062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:56.120375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a047c-65eb-457c-a6c1-d59749494799 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:56.124399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:56.125043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:56.125326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:56.125595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:56.180263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:56.181721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:56.191114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1142/2280] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:11:57.547482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:11:57.548879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:11:57.560715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a012e13-e29d-4a42-b038-a8f61eba58e4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:11:57.564559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:11:57.565210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:11:57.565500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:11:57.565778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:11:57.574926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:11:57.576351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2ebf-4ebd-4a20-ba5f-3222ff0d63e4 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:11:57.585716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1139/2281] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:11:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:02.877326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:02.878913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:02.932143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:02.947964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fed552-f795-4339-be53-6b774049a83d service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:02.952217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:02.952837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:02.953259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:02.953531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:02.955461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:03.020503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:03.021976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a8f1f-bd8a-4a45-ae97-3b82a67de7ad service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:03.032827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1143/2282] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:12:07.581535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=60 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:10.211498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:10.212083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:10.212179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:10.213832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1140/2283] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:12:10.246370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:10.246900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:10.247107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:10.267233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:10.268425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:10.337347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1144/2284] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:12:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:10.348748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:10.350525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:10.367289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72cf49-1870-4859-a871-1ead6bb75ca7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:10.372407 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:10.373050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:10.373908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:10.374185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:10.382649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:10.384235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:10.394368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1141/2285] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:10.800649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:10.863443 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c86a64-6304-4dd8-89f0-6dab290ce01e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:10.867968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:10.868598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:10.868874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:10.869181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] PATH_INFO: `/v3/users/f49c48be0bf74bc59165657b8cbf7de8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:10.870330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49c48be0bf74bc59165657b8cbf7de8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:10.872239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:10.907412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:10.920413 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49c48be0bf74bc59165657b8cbf7de8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:10.932087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49c48be0bf74bc59165657b8cbf7de8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:10.932755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49c48be0bf74bc59165657b8cbf7de8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:10.939584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49c48be0bf74bc59165657b8cbf7de8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:10.946987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49c48be0bf74bc59165657b8cbf7de8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:10.954018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1145/2286] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:12:10 2022] DELETE /identity/v3/users/f49c48be0bf74bc59165657b8cbf7de8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:11.119935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:11.131418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f11cf-08fe-4b5b-b76c-a0421dc95b47 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:11.193738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:11.194300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.194583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.194857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] PATH_INFO: `/v3/projects/494bac11d92540c1aa4a72da53f2d265` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.195996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494bac11d92540c1aa4a72da53f2d265)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:11.197951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:11.244165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494bac11d92540c1aa4a72da53f2d265', 'request_id': 'req-b2622000-a3c7-44fa-9a26-934e7ca74231'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.250931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494bac11d92540c1aa4a72da53f2d265', 'request_id': 'req-b2622000-a3c7-44fa-9a26-934e7ca74231'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.257397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494bac11d92540c1aa4a72da53f2d265', 'request_id': 'req-b2622000-a3c7-44fa-9a26-934e7ca74231'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.270592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] The token cache is being invalidate because project 494bac11d92540c1aa4a72da53f2d265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:12:11.270884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.271514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.273070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1142/2287] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:12:11 2022] DELETE /identity/v3/projects/494bac11d92540c1aa4a72da53f2d265 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:11.277001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:11.278436 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:11.291363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95370a6c-f49c-403a-ae65-415399d54b8f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:11.352248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:11.352901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.353218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.353500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] PATH_INFO: `/v3/users/1b42f5aee6e14f51a669a375d7dd1b31` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.354688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b42f5aee6e14f51a669a375d7dd1b31)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:11.356621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:11.405515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b42f5aee6e14f51a669a375d7dd1b31'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.417349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b42f5aee6e14f51a669a375d7dd1b31'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.418046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b42f5aee6e14f51a669a375d7dd1b31'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.426798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b42f5aee6e14f51a669a375d7dd1b31'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.435245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fbf86ab4-16c5-42c0-ad66-09e53bd3d996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b42f5aee6e14f51a669a375d7dd1b31'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.442670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1146/2288] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:12:11 2022] DELETE /identity/v3/users/1b42f5aee6e14f51a669a375d7dd1b31 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:11.630145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:11.631810 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:11.648413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2622000-a3c7-44fa-9a26-934e7ca74231 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:11.712941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:11.713690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.714044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.714325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] PATH_INFO: `/v3/projects/7f01f53583bb4edc917aa64a50746996` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.715603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f01f53583bb4edc917aa64a50746996)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:11.718174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:11.743398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:11.768893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f01f53583bb4edc917aa64a50746996', 'request_id': 'req-50ef9a40-4a40-44a5-8491-5d0d709d7665'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.774475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f01f53583bb4edc917aa64a50746996', 'request_id': 'req-50ef9a40-4a40-44a5-8491-5d0d709d7665'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.781156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f01f53583bb4edc917aa64a50746996', 'request_id': 'req-50ef9a40-4a40-44a5-8491-5d0d709d7665'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.793233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] The token cache is being invalidate because project 7f01f53583bb4edc917aa64a50746996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:12:11.793516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.794131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-50ef9a40-4a40-44a5-8491-5d0d709d7665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:11.796186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1143/2289] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:12:11 2022] DELETE /identity/v3/projects/7f01f53583bb4edc917aa64a50746996 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:11.801082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.801368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.801639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.821277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:11.822460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:11.845437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:11.883135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1147/2290] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:11.891566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.891857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.892133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.912247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:11.913500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:11.922041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1144/2291] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:11.926167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:11.935815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3cec0-ad8d-4693-b49f-4fda4183cf0d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:11.941188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:11.942696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.942983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.943396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.944110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:12:11.944419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:11.946104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:11.964816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1148/2292] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:12:11.966172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:11.978390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca205c3-7371-4f36-9632-5d2461a6684b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:11.984141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:11.984767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:11.985550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:11.985550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:11.986048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:11.987971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.018715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1145/2293] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:12:11 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:12:12.024209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:12.035181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96a782-ddf4-4748-8fd2-53123cd41182 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:12.093069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:12.093699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.094066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.094350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.095033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:12.097046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.118593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1149/2294] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:12:12 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:12:12.122758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:12.134144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e87a1-eca7-46c9-a59d-d3e6243410d9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:12.139544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:12.140183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.140467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.140744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.141510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:12.143114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.153035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1146/2295] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:12.157511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:12.167454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeac8fd-b39d-4f1b-8763-48e60b4dfe1d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:12.173613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:12.174222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.174506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.174780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.175469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:12.177130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b57215-6e64-4b95-b5a2-1e2fad2ea0c1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.188149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1150/2296] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:12.192569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:12.202665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199dda1-8d7b-4396-8ace-ea0581b021e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:12.207322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:12.207977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.208261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.208541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] PATH_INFO: `/v3/projects/e414f9d75f0d4d089d37409da1debf6c/users/bb21c6b2d833451eae04bbe153aaf394/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.223952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e414f9d75f0d4d089d37409da1debf6c, user_id=bb21c6b2d833451eae04bbe153aaf394, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:12.226608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.240015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1147/2297] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:12:12 2022] PUT /identity/v3/projects/e414f9d75f0d4d089d37409da1debf6c/users/bb21c6b2d833451eae04bbe153aaf394/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:12.250746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e91d-7716-42ed-9fcf-4274be966b76 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.251036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e91d-7716-42ed-9fcf-4274be966b76 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.251316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8e91d-7716-42ed-9fcf-4274be966b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.278026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aac8e91d-7716-42ed-9fcf-4274be966b76 None None] MFA Rules not processed for user `bb21c6b2d833451eae04bbe153aaf394`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:12.279232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aac8e91d-7716-42ed-9fcf-4274be966b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:12.352018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1151/2298] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:12.363183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:12.365008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:12.422447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78fcf-6e8c-45ab-8473-7265b65b3631 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:12.428583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:12.429658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:12.429949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:12.430228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:12.438886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:12.440419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:12.449699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1148/2299] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:22.349026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:22.445620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:39.247655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:39.248159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:39.248278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:39.267780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:39.269010 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:39.323770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1152/2300] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:12:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:12:39.369416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:39.370930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:39.439227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0eabb5-77e3-47bf-ba18-28277b7b66d3 service glance] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:39.444417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:39.445075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:39.445392 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:39.445681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:39.454828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:39.456426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c781f-f7d8-4fff-b97f-0c771f4b2b02 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:39.464398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1149/2301] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:12:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:12:42.831419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:42.910623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a72ce-884a-4258-986c-ae2dbc9b9fcb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:42.915552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:42.916205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:42.916491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:42.916767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] PATH_INFO: `/v3/users/bb21c6b2d833451eae04bbe153aaf394` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:42.918129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb21c6b2d833451eae04bbe153aaf394)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:42.920192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:42.980920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb21c6b2d833451eae04bbe153aaf394'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:42.995930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb21c6b2d833451eae04bbe153aaf394'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:42.996762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb21c6b2d833451eae04bbe153aaf394'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.004739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb21c6b2d833451eae04bbe153aaf394'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.014487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb21c6b2d833451eae04bbe153aaf394'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.023495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1153/2302] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:12:42 2022] DELETE /identity/v3/users/bb21c6b2d833451eae04bbe153aaf394 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:43.028347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:43.028945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:43.028945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:43.059619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:43.061108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:43.173658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1150/2303] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:43.183400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:43.185152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:43.217780 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:43.256157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe5251-bc3c-45b4-8274-0429d19e01e9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:43.265775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:43.266992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:43.267407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:43.267797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:43.281333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:43.283114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48caa785-1141-4d02-b91d-89223169e02c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:43.295522 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1154/2304] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:43.627682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:43.629417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:43.656531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e19a-46fa-402c-90a4-ab929d9783ff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:43.672742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:43.673918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:43.674895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:43.675371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] PATH_INFO: `/v3/projects/e414f9d75f0d4d089d37409da1debf6c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:43.676746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] RBAC: Authorizing `identity:delete_project(project_id=e414f9d75f0d4d089d37409da1debf6c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:43.678954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:43.745652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e414f9d75f0d4d089d37409da1debf6c', 'request_id': 'req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.752274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e414f9d75f0d4d089d37409da1debf6c', 'request_id': 'req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.760183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e414f9d75f0d4d089d37409da1debf6c', 'request_id': 'req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.781889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] The token cache is being invalidate because project e414f9d75f0d4d089d37409da1debf6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:12:43.781889 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.782608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1e8bdb0c-e53d-4d53-a2cd-06bb59f7e4de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:43.784422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1151/2305] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:12:43 2022] DELETE /identity/v3/projects/e414f9d75f0d4d089d37409da1debf6c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:43.790770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:43.791062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:43.791395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:43.838061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:43.839433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:43.937239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1155/2306] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:43.947387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:43.947768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:43.948132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:43.971665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:43.973388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:43.988116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:43.990855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1152/2307] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:12:43.999336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9871a-7613-41ca-a014-b81992dd9f99 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.004696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.005481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.005902 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.006185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.006881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:12:44.007213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.008813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.023202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1156/2308] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:12:44.027319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.038023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d193185-0c83-409a-a41f-763979d6aebd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.045273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.045997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.047050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.047562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.048290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.050377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.055563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:44.089511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1153/2309] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:12:44.096132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.107375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760daae-4c35-4669-892f-42093a0112c2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.183238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.184014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.184014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.184228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.184874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.186739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.209814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1157/2310] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:12:44 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:12:44.213735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.224665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d624-32e8-4346-b846-b75dde29c06a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.229375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.229986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.230273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.230706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.231596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.233275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.243939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1154/2311] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:44.248083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.262207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12838541-f72e-4c89-88d1-744baaf75db1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.270506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.271129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.271406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.271675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.272371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.274910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9ba63-add5-4504-a7bc-11cddbd652a2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.294959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1158/2312] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:44.295773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.308817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0829ce-3179-42bb-8f4a-44f00fa887ac admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.322766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.323667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.323973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.324166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] PATH_INFO: `/v3/projects/0364a4679f1447a78e08428f08658650/users/abdd19dbed75471089555cdbc3437ae8/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.342338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] RBAC: Authorizing `identity:create_grant(project_id=0364a4679f1447a78e08428f08658650, user_id=abdd19dbed75471089555cdbc3437ae8, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.346508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.363266 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1155/2313] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:12:44 2022] PUT /identity/v3/projects/0364a4679f1447a78e08428f08658650/users/abdd19dbed75471089555cdbc3437ae8/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:44.374845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.375287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.375747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.409673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] MFA Rules not processed for user `abdd19dbed75471089555cdbc3437ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:44.411306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:44.435188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:44.486697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1159/2314] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:44.499017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:44.501653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:44.572295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38060f89-60bc-4510-8635-c646c49718df admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:44.576255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:44.577272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:44.578354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:44.579571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:44.592268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:44.594537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17df9-3ae5-47a5-9160-915ab4eb1985 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:44.607734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1156/2315] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:48.339896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:48.422218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593b48-81c4-4f0c-9920-84b15f7c2646 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:48.427338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:48.427984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:48.428264 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:48.428536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] PATH_INFO: `/v3/users/abdd19dbed75471089555cdbc3437ae8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:48.433769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdd19dbed75471089555cdbc3437ae8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:48.436044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:48.493717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdd19dbed75471089555cdbc3437ae8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:48.509626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdd19dbed75471089555cdbc3437ae8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:48.510383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdd19dbed75471089555cdbc3437ae8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:48.519832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdd19dbed75471089555cdbc3437ae8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:48.529621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdd19dbed75471089555cdbc3437ae8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:48.542111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1160/2316] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:12:48 2022] DELETE /identity/v3/users/abdd19dbed75471089555cdbc3437ae8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:12:48.544039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:48.544577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:48.544577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:48.569892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:48.571146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:48.643426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1157/2317] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:48.651994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:48.653618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:48.711550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:48.730308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64c973-1972-4eba-888e-8ed0cd368da7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:48.743464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:48.744115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:48.744398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:48.744671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:48.760254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:48.761736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80b351-40f4-46f6-be56-05cd8f0751b5 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:48.788585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1161/2318] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:12:49.096729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.101602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:49.115778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661e670-3ec0-4777-bc93-6f7668a22754 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.121410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.122612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.122974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.123405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] PATH_INFO: `/v3/projects/0364a4679f1447a78e08428f08658650` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.124605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0364a4679f1447a78e08428f08658650)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.127208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.174958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:49.176232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0364a4679f1447a78e08428f08658650', 'request_id': 'req-c772e6f4-b7b6-48d6-983e-39f8ad968eba'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:49.183023 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0364a4679f1447a78e08428f08658650', 'request_id': 'req-c772e6f4-b7b6-48d6-983e-39f8ad968eba'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:49.189614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0364a4679f1447a78e08428f08658650', 'request_id': 'req-c772e6f4-b7b6-48d6-983e-39f8ad968eba'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:49.202557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] The token cache is being invalidate because project 0364a4679f1447a78e08428f08658650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:12:49.202851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:49.203549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c772e6f4-b7b6-48d6-983e-39f8ad968eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:12:49.205152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1158/2319] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:12:49 2022] DELETE /identity/v3/projects/0364a4679f1447a78e08428f08658650 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:49.210061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.210345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.210621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.230417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:49.231610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:49.289920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1162/2320] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:49.357123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.357441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.357718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.377528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:49.378718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:49.386472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1159/2321] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:49.391277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.400882 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378ef75-6417-4423-b8de-b07c0b78ff97 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.405449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.406660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.406867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.407135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.407813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:12:49.408112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.409732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.420897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1163/2322] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:12:49.426067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.434979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa33cb-4a14-4237-a923-72400723cccc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.439586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.440183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.440464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.440737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.441439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.443585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.471910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1160/2323] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:12:49 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:12:49.475977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.486315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f621d-b51a-490c-9644-a85150831e88 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.551269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.551897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.552183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.552461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.553171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.555147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.576208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1164/2324] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:12:49 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:12:49.581263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.591727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a839-c046-4277-9a0c-67900cb01a48 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.597009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.597619 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.597903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.598181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.598876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.600470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.610367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1161/2325] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:49.614171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.623942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378e1f-8ad9-4901-9f08-bc89f4a0a08f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.629116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.629717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.630001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.630277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.630967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.632554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c55047-30e8-44c1-90a3-92c3f3392d20 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.643574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1165/2326] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:12:49.647676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.658274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eee36-92fb-49a8-ab42-2a1267c03872 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.662914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.663599 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.663877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.664151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] PATH_INFO: `/v3/projects/5acfaf1104da41f8ba35ddbebb79b506/users/5cc7b5ba908541cf929eaf10ba24e465/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.678413 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acfaf1104da41f8ba35ddbebb79b506, user_id=5cc7b5ba908541cf929eaf10ba24e465, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.681026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.694450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1162/2327] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:12:49 2022] PUT /identity/v3/projects/5acfaf1104da41f8ba35ddbebb79b506/users/5cc7b5ba908541cf929eaf10ba24e465/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:12:49.699378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33471-7f9e-4d01-9949-4c675ed0b913 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.699671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33471-7f9e-4d01-9949-4c675ed0b913 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.699951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33471-7f9e-4d01-9949-4c675ed0b913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.719544 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e3b33471-7f9e-4d01-9949-4c675ed0b913 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:49.720744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3b33471-7f9e-4d01-9949-4c675ed0b913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:49.764263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:49.780496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1166/2328] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:49.790917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:49.792735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:49.857121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be9f59-b9e9-464f-8383-48ca79c2a802 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:49.860811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:49.861460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:49.861740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:49.862014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:49.871096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:49.872549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:49.884232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1163/2329] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:50.569171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848336-0b0e-4aac-8788-44d98b8ca613 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:50.569713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848336-0b0e-4aac-8788-44d98b8ca613 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:50.569713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0848336-0b0e-4aac-8788-44d98b8ca613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:50.603982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e0848336-0b0e-4aac-8788-44d98b8ca613 None None] MFA Rules not processed for user `5cc7b5ba908541cf929eaf10ba24e465`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:50.605510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e0848336-0b0e-4aac-8788-44d98b8ca613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:50.682640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1167/2330] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:50.694672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:50.696092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:50.754351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953704f6-6445-4598-8939-81e4c2b849cb service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:50.758024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:50.758639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:50.758923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:50.759197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:50.767691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:50.769191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6c36e-1caf-46be-80df-8dd1206b7ed4 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:50.778141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1164/2331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:52.130744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ae3b-2edb-4486-81e8-cc65b9a6e5eb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:52.131320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ae3b-2edb-4486-81e8-cc65b9a6e5eb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:52.131320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ae3b-2edb-4486-81e8-cc65b9a6e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:52.150035 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8d66ae3b-2edb-4486-81e8-cc65b9a6e5eb None None] MFA Rules not processed for user `5cc7b5ba908541cf929eaf10ba24e465`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:52.151144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8d66ae3b-2edb-4486-81e8-cc65b9a6e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:52.158447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1168/2332] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:12:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:54.224135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.224696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.224696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.226246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1165/2333] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:12:54.232273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f317bd-7167-4bc9-ba32-8315147096c6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.232720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f317bd-7167-4bc9-ba32-8315147096c6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.232801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f317bd-7167-4bc9-ba32-8315147096c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.255294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-45f317bd-7167-4bc9-ba32-8315147096c6 None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:54.256843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-45f317bd-7167-4bc9-ba32-8315147096c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:54.337250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1169/2334] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:12:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:12:54.356297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:54.358395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:54.380738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:12:54.420939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1addb-f838-4f2e-80c6-fa4eac03348e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:54.424729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:54.425382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.425671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.425943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.436334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:54.438374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cdd14-32bd-484e-9cf1-efba35cdf6b4 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:54.449427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1166/2335] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:54.521640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45175074-3dea-46fb-af6e-93be8cd80c7d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.521931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45175074-3dea-46fb-af6e-93be8cd80c7d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.522207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45175074-3dea-46fb-af6e-93be8cd80c7d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.523640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1170/2336] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:12:54.528503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f409d5a-a34b-4fcf-8c85-7b1c7269b9cf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.528880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f409d5a-a34b-4fcf-8c85-7b1c7269b9cf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.529261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f409d5a-a34b-4fcf-8c85-7b1c7269b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.563625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9f409d5a-a34b-4fcf-8c85-7b1c7269b9cf None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:54.565460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9f409d5a-a34b-4fcf-8c85-7b1c7269b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:54.574750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1167/2337] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:12:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:12:54.587133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45175074-3dea-46fb-af6e-93be8cd80c7d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:54.598151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45175074-3dea-46fb-af6e-93be8cd80c7d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:54.602003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:54.602617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.602901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.603178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.610963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:54.612451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6f22e-537f-4597-928d-8f8a668b4dbb service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:54.621805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1171/2338] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:12:54.706147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e4ee6-5b5d-4914-b821-e46b06d91bef None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.706433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e4ee6-5b5d-4914-b821-e46b06d91bef None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.706711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e4ee6-5b5d-4914-b821-e46b06d91bef None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.708067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1168/2339] 10.222.0.65 () {54 vars in 921 bytes} [Wed Jul 27 20:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 20:12:54.713756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2f80-b08e-4df9-aa64-a3d918b3ae6a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.714039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2f80-b08e-4df9-aa64-a3d918b3ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.714313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e2f80-b08e-4df9-aa64-a3d918b3ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.736990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c10e2f80-b08e-4df9-aa64-a3d918b3ae6a None None] MFA Rules not processed for user `056e9b8fcfc84ee68a018f331e805582`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:12:54.738176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c10e2f80-b08e-4df9-aa64-a3d918b3ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:12:54.749341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1172/2340] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 20:12:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:12:54.760349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e4ee6-5b5d-4914-b821-e46b06d91bef None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:12:54.772282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e4ee6-5b5d-4914-b821-e46b06d91bef None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:12:54.776267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:12:54.776945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:12:54.777333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:12:54.777677 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:12:54.788651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:12:54.790113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:12:54.802276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1169/2341] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:13:00.778360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:00.778831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:00.778933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:00.798316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:00.799509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:00.846752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:00.860656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1173/2342] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:13:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:00.872443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:00.873891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:00.945053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941b80e-f76f-4eab-8aed-d9f3b7f72622 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:00.950043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:00.950680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:00.950946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:00.951228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:00.959617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:00.961246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:00.969362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1170/2343] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:13:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:13:08.346796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:08.348278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:08.361564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efad8a-387d-49c7-a9e5-20b3fc53dc24 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:08.365341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:08.366042 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:08.366330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:08.366608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:08.368347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:08.382540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:08.383970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:08.394923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1174/2344] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:08.493579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:08.505013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca533d7f-d5f4-4385-b830-300c475d6efa service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:08.517455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:08.518104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:08.518371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:08.518648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:08.520450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:08.536560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:08.538136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:08.550391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1171/2345] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:18.389935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:18.544182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=65 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:19.202488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:19.203960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:19.217752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c8c346-5163-42cf-8bfb-2b761f978502 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:19.223956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:19.224564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:19.224849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:19.225162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:19.226884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:19.284205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:19.285826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadffcdf-c30e-43c9-bfba-c7645ae28084 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:19.295710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1175/2346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:29.277025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:29.278454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.290613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=28 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:29.293015 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9dde2-dff1-46b8-8797-fd0686e95073 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:29.298842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:29.299421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:29.299708 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:29.299989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:29.303743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:29.371031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:29.372633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:29.381753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1172/2347] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:31.099671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.100119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.100232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.120241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:13:31.121886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:31.132104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1176/2348] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:13:31.144043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:31.157905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0e22-2dd4-4f8d-a01c-2cd627d66222 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:31.163918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:31.164415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:31.164694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:31.164986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:31.173723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:31.175280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:31.184731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1173/2349] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:13:39.568127 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:13:39.579136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1dbc2-b0fe-42db-9f78-6804dbc4fa09 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:13:39.588045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:13:39.588739 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:13:39.588981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:13:39.589256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:13:39.594300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:13:39.605124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:13:39.606612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9e67-c24a-4828-a492-df948ab5999e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:13:39.614807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1177/2350] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:13:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:13:41.181376 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=34 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:13:49.611122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:06.960320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:14:06.972150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c94f86-955e-483b-a129-f7206d3559b2 service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:14:06.978122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:14:06.978724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:06.979007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:06.979285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:06.981057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:07.038343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:14:07.039948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:14:07.049217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1174/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:14:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:14:17.044969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=28 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:14:59.559842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:14:59.560399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:14:59.560399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:14:59.580741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] MFA Rules not processed for user `a48eaef4d70b4116bd307374d819bca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:14:59.581999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:14:59.589040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1178/2352] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:15:07.172738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:07.184944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7044c-7739-415f-81df-6ce62e19bef6 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:07.191532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:07.194638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:07.194638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:07.194638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:07.195025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:07.207131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:07.209289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:07.219681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1175/2353] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:08.181312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:08.194666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53b2ba-8c8f-4006-aaa7-2993b00a2720 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:08.200081 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:08.200687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:08.201053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:08.201379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:08.204446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:08.264222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:08.265777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:08.275756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1179/2354] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:08.850390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:08.863761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751da62-87a7-450b-ae23-eb3a2c887882 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:08.872438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:08.873629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:08.874082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:08.874583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:08.878711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:08.901507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:08.903865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:08.932596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1176/2355] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:14.563390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:14.576559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e7c02-8769-4a48-a355-b709bbfcb373 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:14.583177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:14.583763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:14.584168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:14.584469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:14.586335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:14.598910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:14.600517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:14.611369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1180/2356] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:15:18.924435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=12 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:24.607351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=37 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:15:56.734005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:15:56.745630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549074-0513-4901-89ad-a332de2a8dc0 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:15:56.751188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:15:56.751792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:15:56.752076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:15:56.752354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:15:56.754142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:15:56.812206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:15:56.813831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:15:56.823488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1177/2357] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:16:06.818800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=28 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:17:18.656684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:17:18.658144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:17:18.797999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb293d-f536-4302-a94d-c6381012ee7d service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:17:18.803545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:17:18.804212 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:17:18.804496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:17:18.804772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:17:18.806563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:17:18.818276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:17:18.819846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:17:18.829500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1181/2358] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:17:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:17:28.825454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=63 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:19:00.074383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:00.075829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:19:00.130442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae558c7b-aea1-4af8-b521-391ac0de565f service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:00.134346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:00.134968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:00.135246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:00.135519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:00.152798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:00.154260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:00.175087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1178/2359] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:19:00.183033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:00.192385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cef88-b703-4d9e-a7f2-f1167cf971ad service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:00.196141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:00.196740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:00.197057 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:00.197334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:00.205703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:00.207112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:00.235560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1182/2360] 10.222.0.65 () {60 vars in 1396 bytes} [Wed Jul 27 20:19:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:00.280364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:00.290565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940c92e-c546-4006-9371-e365968fb335 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:00.294504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:00.295132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:00.295417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:00.295696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:00.307598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:00.309217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:00.322279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1179/2361] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:00.329561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:00.339434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17ab39-c873-41d2-8a53-d7530ba91ccc service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:00.343272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:00.343880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:00.344166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:00.344438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:00.358283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:00.359724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:00.388677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1183/2362] 10.222.0.65 () {60 vars in 1396 bytes} [Wed Jul 27 20:19:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:00.477982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:00.486859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfeaa9-d498-4a88-925c-75b819c15f0f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:00.491196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:00.491815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:00.492119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:00.493031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:00.502249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:00.503725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:00.514463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1180/2363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:10.385531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=26 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:19:10.510105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=43 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:19:12.436013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:12.444650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2d950-d9aa-45f9-b105-47355663f4cd service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:12.449255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:12.449943 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:12.450286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:12.450656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:12.459414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:12.460831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:12.487106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1184/2364] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:18.781114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:19:18.800079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb9bda-1de0-405e-8306-a77fb54505fb service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:19:18.803705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:19:18.804310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:19:18.804592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:19:18.804870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:19:18.812431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:19:18.813899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:19:18.822648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1181/2365] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:19:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:19:22.484476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=13 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:19:28.818471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=10 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:21:00.755825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:21:00.767603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c696a8-f440-446a-913d-5a7fbb9b0817 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:21:00.773225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:21:00.773842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:21:00.774128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:21:00.774405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:21:00.784390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:21:00.790696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:21:00.795845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:21:00.807918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2917-4d26-4064-b6e9-aabef51cd4b0 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:21:00.815245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:21:00.816292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:21:00.816675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:21:00.817215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:21:00.819967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1185/2366] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:21:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:21:00.827544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:21:00.829379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:21:00.870226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1182/2367] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:21:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:21:10.816596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=13 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:21:10.857786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=13 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:22:23.026871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:22:23.028248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:22:23.042517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee67e-eb1d-447b-a763-aee001d49338 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:22:23.047420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:22:23.048017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:22:23.048294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:22:23.048565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:22:23.057323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:22:23.058897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:22:23.068895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1186/2368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:22:33.064342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:24:26.684181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:24:26.685802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:24:26.753655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408353c-8db2-4e79-9067-b9f53a8eb962 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:24:26.757232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:24:26.757842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:24:26.758127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:24:26.758404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:24:26.760220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:24:26.769984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:24:26.771399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:24:26.781982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1183/2369] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:24:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:24:36.775992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=31 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:01.067552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.125062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c88b2-ef95-445d-812d-4a7ac000cf92 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.128693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.129345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.129628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.129904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.131776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.217600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:01.219041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:01.230424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1187/2370] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:01.237420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.247237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c255-44d3-471f-add7-9bc5ed2040cd service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.250817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.251412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.251691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.251973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.253976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.327951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:01.329527 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:01.339063 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1184/2371] 10.222.0.65 () {60 vars in 1396 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:01.383204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.393128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40245e7b-faa6-4b74-89ca-c9940118532b service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.397824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.398437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.398714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.398988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.400865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.415044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:01.417128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:01.428731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1188/2372] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:01.436498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.445210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15ce4d-679b-4dce-bf76-5353e8362718 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.449962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.450627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.450917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.451196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.453139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.526506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:01.528453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:01.537803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1185/2373] 10.222.0.65 () {60 vars in 1396 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:01.855468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.857134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.871900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34fca9-3f74-4734-959a-bfff1493c9d7 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.877783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.878394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.878678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.878961 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.880984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.949146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:01.951524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.969209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:01.972164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5ad9d-3991-4a4c-85a6-6a4019998bc6 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:01.972322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:01.973968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:01.980268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:01.981071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:01.981560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:01.981975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:01.984578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:01.994555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1189/2374] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:02.060561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:02.062284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:02.071269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1186/2375] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:11.991683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:12.067064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:15.921732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:15.935578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315612-516c-4a5a-b7a3-dddf13198549 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:15.941469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:15.942182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:15.942469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:15.943318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:15.947327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:15.961318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:15.962920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:15.975248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1190/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:25:16.998806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:25:17.016259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f9d61-0955-4545-b996-139ab372ba48 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:25:17.023734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:25:17.024650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:25:17.025109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:25:17.025486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:25:17.027885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:25:17.062828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:25:17.065559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:25:17.094037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1187/2377] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:25:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:25:25.970150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:25:27.082494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:26:01.656302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:26:01.667405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4df7-b45a-4c30-b071-45ace4b848be service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:26:01.673295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:26:01.673907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:26:01.674193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:26:01.674469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:26:01.676315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:26:01.744721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:26:01.746847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:26:01.755520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1191/2378] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:26:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:26:11.751498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:27:01.880020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:27:01.889981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16071805-ab76-445c-9b7b-c3ed567b5e17 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:27:01.893748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:27:01.894362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:27:01.894646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:27:01.894930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:27:01.896804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:27:01.968242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:27:01.969696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:27:01.978598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1188/2379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:27:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:27:11.974517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=33 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:27:28.006440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:27:28.008053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:27:28.129584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d5e19-5cf6-468f-b484-0b75b06ef0da service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:27:28.134695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:27:28.135305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:27:28.135589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:27:28.135863 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:27:28.137734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:27:28.149145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:27:28.150756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:27:28.160909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1192/2380] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:27:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:27:38.156825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=54 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:29:33.698745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:29:33.728797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979f21a-81da-4673-82d7-133d2310d278 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:29:33.732565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:29:33.733958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:29:33.734238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:29:33.734511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:29:33.741826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:29:33.743253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:29:33.751956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1189/2381] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:29:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:29:43.747725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=15 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:00.996381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:01.034650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdbb74-01a2-49aa-aeee-2800d06a1c48 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:01.040216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:01.040829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:01.040951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:01.041392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:01.041392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:01.050795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:01.052373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:01.053885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f68774-acf2-4ad4-8c7d-32b1218e306c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:01.059341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:01.059939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:01.060217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:01.060493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:01.074595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:01.076219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:01.081053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1193/2382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:01.105334 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1190/2383] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:30:01.196146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:01.205859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b36ff-a9de-4bd8-8a81-7bd4432e7eab service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:01.209473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:01.210076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:01.210355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:01.210665 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:01.220746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:01.222228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:01.233437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1194/2384] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:02.339060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:30:02.340545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:30:02.365262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe451c7-5091-45c8-9aa0-e7ff35dcfa92 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:30:02.368868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:30:02.369518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:30:02.369801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:30:02.370080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:30:02.380639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:30:02.382127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:30:02.393013 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1191/2385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:30:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:30:11.228176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=31 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:30:12.388043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=23 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:00.728583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:00.739759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d9b7-2f49-43fe-ad0c-b72a1388e5b4 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:00.745827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:00.746423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:00.746705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:00.746983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:00.754647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:00.756237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:00.765643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1195/2386] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:02.408603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:02.410176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:02.443524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390756c-a2aa-4dc4-a7ba-418d70dc8b7c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:02.447818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:02.448427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:02.448706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:02.449004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:02.457367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:02.458782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:02.468484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1192/2387] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:31:02.555874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:02.565190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa4115-74c2-4490-9df4-97f1d5945194 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:02.568773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:02.569408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:02.569692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:02.569972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:02.579152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:02.580578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:02.607462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1196/2388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:12.464181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=15 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:12.604884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=19 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:16.577448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:16.587225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace8d9-265b-44ea-bd44-9bab48f7b372 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:16.591548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:16.592177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:16.592454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:16.592723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:16.594514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:16.679062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:16.680491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:16.699712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1193/2389] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:20.215776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:20.231398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3471a-bd1b-4508-a8f4-9b182757609e service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:20.240353 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:20.241162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:20.241444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:20.241981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:20.244680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:20.261401 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:20.264351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:20.292854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1197/2390] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:23.566080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:23.588267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8a9fa-82fb-40dd-88aa-92e4bd125986 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:23.598082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:23.599057 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:23.599564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:23.600073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:23.602388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:23.630521 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:23.631482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:23.632040 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:23.633630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:23.648926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1194/2391] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:31:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:31:23.649457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713961e-5ef4-43b5-90f6-19e64e3a9b8f service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:23.654897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:23.655558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:23.655835 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:23.656265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] PATH_INFO: `/v3/users/5f8ecfcb241d468685b22c3f883b6623` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:23.659456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8ecfcb241d468685b22c3f883b6623)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:23.659625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:23.719454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8ecfcb241d468685b22c3f883b6623'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:23.734205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8ecfcb241d468685b22c3f883b6623'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:23.734923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8ecfcb241d468685b22c3f883b6623'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:23.741663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8ecfcb241d468685b22c3f883b6623'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:23.749551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8ecfcb241d468685b22c3f883b6623'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:23.755470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1198/2392] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:31:23 2022] DELETE /identity/v3/users/5f8ecfcb241d468685b22c3f883b6623 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:31:23.936132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:23.954039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0dc9d-75ee-493e-9ec5-0817db541a47 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:24.022158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:24.022898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:24.023705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:24.024218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] PATH_INFO: `/v3/projects/87d5b073df6b48908d97ae613731adc1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:24.036807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d5b073df6b48908d97ae613731adc1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:24.039469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:24.100397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:24.105665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d5b073df6b48908d97ae613731adc1', 'request_id': 'req-1df5f4ef-d9dc-4161-bb29-98cfad466b66'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.111588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d5b073df6b48908d97ae613731adc1', 'request_id': 'req-1df5f4ef-d9dc-4161-bb29-98cfad466b66'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.118378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d5b073df6b48908d97ae613731adc1', 'request_id': 'req-1df5f4ef-d9dc-4161-bb29-98cfad466b66'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.130149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] The token cache is being invalidate because project 87d5b073df6b48908d97ae613731adc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:31:24.130440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.131065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.132571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1195/2393] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:31:23 2022] DELETE /identity/v3/projects/87d5b073df6b48908d97ae613731adc1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:31:24.137303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:24.138698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:24.152506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b695-1f03-4e71-a3e2-6a21d6498216 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:24.207258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:24.207888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:24.208167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:24.208442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] PATH_INFO: `/v3/users/6bd8078f37294bb29879b3b57f4c3466` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:24.221651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd8078f37294bb29879b3b57f4c3466)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:24.223581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:24.268636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd8078f37294bb29879b3b57f4c3466'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.270551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:24.282614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd8078f37294bb29879b3b57f4c3466'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.283275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd8078f37294bb29879b3b57f4c3466'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.290047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd8078f37294bb29879b3b57f4c3466'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.297204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd8078f37294bb29879b3b57f4c3466'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.303440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1199/2394] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:31:24 2022] DELETE /identity/v3/users/6bd8078f37294bb29879b3b57f4c3466 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:31:24.480646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:31:24.482387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:31:24.495868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5f4ef-d9dc-4161-bb29-98cfad466b66 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:31:24.558506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:31:24.559141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:31:24.559419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:31:24.559689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] PATH_INFO: `/v3/projects/6f94b946276c470b8483c8d909cafacf` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:31:24.570283 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f94b946276c470b8483c8d909cafacf)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:31:24.572228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:31:24.617947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f94b946276c470b8483c8d909cafacf', 'request_id': 'req-c6ea78de-f8da-465e-b996-39abeeefda0d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.624180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f94b946276c470b8483c8d909cafacf', 'request_id': 'req-c6ea78de-f8da-465e-b996-39abeeefda0d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.630626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f94b946276c470b8483c8d909cafacf', 'request_id': 'req-c6ea78de-f8da-465e-b996-39abeeefda0d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.642894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] The token cache is being invalidate because project 6f94b946276c470b8483c8d909cafacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:31:24.643293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.643940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:31:24.645607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1196/2395] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:31:24 2022] DELETE /identity/v3/projects/6f94b946276c470b8483c8d909cafacf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:31:34.300025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=11,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:31:34.639927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=11,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:32:01.474745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:32:01.476292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:32:01.552955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7586e-17b0-4a06-9c55-b43b59c08e71 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:32:01.617352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:32:01.617979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:32:01.618263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:32:01.618545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:32:01.620318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:32:01.686513 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:32:01.688319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:32:01.706685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1200/2396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:32:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:32:11.704206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:32:32.029994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:32:32.031721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:32:32.101759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea78de-f8da-465e-b996-39abeeefda0d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:32:32.106897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:32:32.107519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:32:32.107800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:32:32.108073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:32:32.110504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:32:32.121851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:32:32.123430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:32:32.133847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1197/2397] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:32:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:32:42.129678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=32 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:33:20.040806 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:33:20.041861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:33:20.061734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3a69e-3306-4dbb-8830-a86c057d11ac service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:33:20.076531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:33:20.077290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:33:20.077944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:33:20.078656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:33:20.080786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:33:20.098942 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:33:20.100627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:33:20.113022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1201/2398] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:33:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:33:30.107921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:35:00.670021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:35:00.671508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:00.740350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e7088-ff74-4872-9d65-0b508cb09a12 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:35:00.743983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:35:00.744598 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:35:00.744882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:35:00.745192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:35:00.746969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:00.760268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:35:00.761719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:35:00.772570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1198/2399] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:35:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:35:01.946921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:35:01.948467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:01.960143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f69697-bbca-4365-adaf-4c961dc96ec8 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:35:01.963697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:35:01.964291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:35:01.964571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:35:01.964852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:35:01.966606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:01.993370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:35:01.994801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:35:02.005694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1202/2400] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:35:03.062202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:35:03.063611 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:03.124076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d29fb-c7ec-4f9f-b67a-fd99a2933a88 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:35:03.127693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:35:03.128870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:35:03.130590 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:35:03.130893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:35:03.132698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:03.216118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:35:03.217714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:35:03.228031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1199/2401] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:35:03.286207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:35:03.296101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb62a-c971-4c56-903c-f470fafcc266 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:35:03.300843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:35:03.301502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:35:03.301782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:35:03.302059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:35:03.303930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:03.374946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:35:03.376381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:35:03.386307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1203/2402] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:35:13.223037 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=84 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:35:13.382134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=44 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:35:38.684835 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:35:38.695869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90be87-e511-40f1-aa07-57685dc515ab service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:35:38.699705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:35:38.700344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:35:38.700645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:35:38.701077 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:35:38.703316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:35:38.713474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:35:38.714958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:35:38.723043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1200/2403] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:35:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:35:48.719336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:02.648580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:02.661427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9c77-7307-4686-a1cb-0f070a9dc112 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:02.666886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:02.667488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:02.667766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:02.668038 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:02.669885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:02.739745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:02.741394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:02.750395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1204/2404] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:03.038049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:36:03.039524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:03.055335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967532e-90fb-47e9-a2a6-9a46e45fab0c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:36:03.060952 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:36:03.061586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:36:03.061865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:36:03.062140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:36:03.063817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:36:03.144445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:36:03.146124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49817d7-4495-44aa-9a5f-3b47b7468310 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:36:03.164156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1201/2405] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:36:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:36:12.746119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:36:13.161587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=41 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:02.036101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:02.047235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7455e-ab58-4ed6-84ff-8a4bddb5d8e5 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:02.052619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:02.053261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:02.053542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:02.053815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:02.061583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:02.063164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:02.072237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1205/2406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:12.068046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:17.645219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:17.645691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:17.645799 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:17.709719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:37:17.711032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:17.783571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1202/2407] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:37:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:37:17.829904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:17.831305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:17.918343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a08ce-e8e3-408d-8b0e-92d1a9b5e96d service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:17.923854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:17.924464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:17.924746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:17.925059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:17.932890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:17.934496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:17.943002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1206/2408] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:37:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:37:18.031193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:18.032726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:18.044282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c01dd-85d5-4b33-9722-398a9b518714 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:18.048516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:18.049234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:18.049585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:18.049868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:18.051873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] Could not find project: 87d5b073df6b48908d97ae613731adc1.: keystone.exception.ProjectNotFound: Could not find project: 87d5b073df6b48908d97ae613731adc1. Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 27 20:37:18.077010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:37:18.078666 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:37:18.080180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 20:37:18.082477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 20:37:18.084487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 87d5b073df6b48908d97ae613731adc1. Jul 27 20:37:18.084487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 20:37:18.084487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1203/2409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:18 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 27 20:37:18.514019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:18.534380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff996c-9d0f-4444-ac70-0e235eb7e1b3 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:18.539916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:18.540692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:18.541019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:18.541540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:18.545092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:18.672064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:18.675431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:18.692265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1207/2410] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:22.019263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:22.035996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12c49d-4068-470a-87f7-a57a74e16bf5 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:22.046660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:22.047447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:22.048239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:22.048517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:22.050328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:22.062252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:22.064488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:22.098427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1204/2411] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:25.560014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:25.561530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:25.577126 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4b718-5a98-40da-8f23-178b153a8fb3 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:25.582808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:25.583442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:25.583722 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:25.583996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] PATH_INFO: `/v3/users/5fa85a53bfbe4ef5a36c01031f9016aa` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:25.598441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa85a53bfbe4ef5a36c01031f9016aa)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:25.600359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:25.648151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa85a53bfbe4ef5a36c01031f9016aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.649279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:25.661875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa85a53bfbe4ef5a36c01031f9016aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.662569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa85a53bfbe4ef5a36c01031f9016aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.669711 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa85a53bfbe4ef5a36c01031f9016aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.678251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa85a53bfbe4ef5a36c01031f9016aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.689967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1208/2412] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:37:25 2022] DELETE /identity/v3/users/5fa85a53bfbe4ef5a36c01031f9016aa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:37:25.867973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:25.880197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce69d7-1a44-4865-bced-8e1804a749d6 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:25.940897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:25.941565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:25.941844 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:25.942118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] PATH_INFO: `/v3/projects/4d888dcf554944adae3e03bc1c24c52b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:25.943264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d888dcf554944adae3e03bc1c24c52b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:25.945223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:25.989353 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:25.993519 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d888dcf554944adae3e03bc1c24c52b', 'request_id': 'req-9050ec49-7cdb-4679-adfc-39d9913045f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:25.999529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d888dcf554944adae3e03bc1c24c52b', 'request_id': 'req-9050ec49-7cdb-4679-adfc-39d9913045f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.006270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d888dcf554944adae3e03bc1c24c52b', 'request_id': 'req-9050ec49-7cdb-4679-adfc-39d9913045f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.018278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] The token cache is being invalidate because project 4d888dcf554944adae3e03bc1c24c52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:37:26.018562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.019176 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.020649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1205/2413] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:37:25 2022] DELETE /identity/v3/projects/4d888dcf554944adae3e03bc1c24c52b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:37:26.024634 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:26.026061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:26.039905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a983985-d2d1-46fc-9aac-b1aa63e4f316 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:26.099414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:26.100067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:26.100348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:26.100624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] PATH_INFO: `/v3/users/46fb4cbeed9b4927a35c8bade8dd12ab` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:26.114352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fb4cbeed9b4927a35c8bade8dd12ab)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:26.116289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:26.161775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fb4cbeed9b4927a35c8bade8dd12ab'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.174120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fb4cbeed9b4927a35c8bade8dd12ab'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.174809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fb4cbeed9b4927a35c8bade8dd12ab'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.181679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fb4cbeed9b4927a35c8bade8dd12ab'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.190729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fb4cbeed9b4927a35c8bade8dd12ab'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.197296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1209/2414] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:37:26 2022] DELETE /identity/v3/users/46fb4cbeed9b4927a35c8bade8dd12ab => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:37:26.426333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:26.427812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:26.442473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050ec49-7cdb-4679-adfc-39d9913045f0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:26.503381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:26.503943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:26.504225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:26.504501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] PATH_INFO: `/v3/projects/bb6f73a4292d4ae4b7bd78a894936879` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:26.514723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6f73a4292d4ae4b7bd78a894936879)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:26.516721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:26.564636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6f73a4292d4ae4b7bd78a894936879', 'request_id': 'req-043b1b3a-a856-4f5d-9ee1-68451cd42c36'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.570205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6f73a4292d4ae4b7bd78a894936879', 'request_id': 'req-043b1b3a-a856-4f5d-9ee1-68451cd42c36'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.577387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6f73a4292d4ae4b7bd78a894936879', 'request_id': 'req-043b1b3a-a856-4f5d-9ee1-68451cd42c36'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.590340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] The token cache is being invalidate because project bb6f73a4292d4ae4b7bd78a894936879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:37:26.590742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.591377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-043b1b3a-a856-4f5d-9ee1-68451cd42c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:37:26.593114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1206/2415] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:37:26 2022] DELETE /identity/v3/projects/bb6f73a4292d4ae4b7bd78a894936879 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:37:36.193431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=70,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:36.587180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:37:36.986429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:37:36.988003 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:37.117775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec9cc1-bed5-43dc-9c4c-a6bf9b7aacb3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:37:37.176981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:37:37.177618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:37:37.177897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:37:37.178178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:37:37.179971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:37:37.191547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:37:37.193178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:37:37.204026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1210/2416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:37:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:37:47.198753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=80 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:39:20.456642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:20.457857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:20.457857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:20.485097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] MFA Rules not processed for user `ea3c5c3007e04391b5b108473da62787`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:39:20.486306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:20.540890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1207/2417] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:39:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:39:20.857695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:39:20.859120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:20.914307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed242f71-260a-463a-b2ce-987dfd687b34 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:39:20.917892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:39:20.918439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:20.918716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:20.918987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:20.920701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:20.929871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:39:20.931282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:39:20.938977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1211/2418] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:39:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:39:21.283975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:39:21.285758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:21.300761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ff44c-25d4-4a05-b8e9-ce9a4038ecfb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:39:21.306150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:39:21.306957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:21.307311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:21.307654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:21.309479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:21.322512 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:39:21.324187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:39:21.337235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1208/2419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:39:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:39:24.083604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:39:24.085716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:24.181406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404aba3-d020-4984-8c4e-260f652b591f service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:39:24.188051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:39:24.188905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:24.189388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:24.189818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:24.192032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:24.206613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:39:24.208531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:39:24.220269 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1212/2420] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:39:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:39:30.080505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:39:30.096094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d61e1-1103-4fac-b838-0aaa9bc129fd service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:39:30.102859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:39:30.103592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:30.103883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:30.104268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:30.106247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:30.120427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:39:30.122180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:39:30.134860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1209/2421] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:39:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:39:34.133965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:39:34.135476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:34.149120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e7609-8d74-4517-8c5c-6ab7819cfeea service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:39:34.154753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:39:34.155365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:39:34.155646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:39:34.155922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:39:34.157736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:39:34.169926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:39:34.171520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:39:34.182698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1213/2422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:39:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:39:40.129985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=40 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:39:44.178812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=66 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:40:00.887691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:00.889122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:00.909868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a7202-dec9-4351-9480-016efd6d6f96 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:00.913452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:00.914047 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:00.914367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:00.914640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:00.916317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:00.929072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:00.931118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:00.941624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1210/2423] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:02.289909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:02.309348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0c78c-c421-4d9f-bf1f-a92307848b6f service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:02.319067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:02.319696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:02.319978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:02.320252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:02.322248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:02.343627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:02.345023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:02.365861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1214/2424] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:03.306269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:03.324880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b31bb-579d-45d8-8b50-3d202ab46258 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:03.328797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:03.329384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:03.329669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:03.329942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:03.331756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:03.409212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:03.410679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:03.432093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1211/2425] 10.222.0.65 () {60 vars in 1396 bytes} [Wed Jul 27 20:40:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:03.498374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:03.510365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8212a-941b-4ed7-b78f-9150b3225888 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:03.516198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:03.516807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:03.517123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:03.517396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:03.519152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:03.591318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:03.592831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:03.611021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1215/2426] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:13.428501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=55 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:40:13.608578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=54 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:40:39.671248 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:40:39.679815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ddcbb5-7cbb-4d19-a252-608ddf420155 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:40:39.683395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:40:39.683996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:40:39.684273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:40:39.684546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:40:39.686296 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:40:39.695411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:40:39.697383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:40:39.705433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1212/2427] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:40:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:40:49.701641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:03.658429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:03.659879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:03.750096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b30cb-12b0-4ccb-a28e-3ca3716de523 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:03.754183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:03.754799 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:03.755083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:03.755437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:03.757133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:03.828845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:03.830328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:03.839838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1216/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:41:04.054946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:41:04.066117 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb0dfb-9485-4efd-9e53-c5dd24fbac9c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:41:04.071762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:41:04.072838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:41:04.073391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:41:04.074549 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:41:04.077977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:41:04.114439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:41:04.117273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:41:04.144044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1213/2429] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:41:13.835751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=57 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:41:14.139471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=14 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:42:42.754658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:42:42.756063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:42:42.770376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd5160-d6cc-4c47-b043-5805fae3c16e service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:42:42.775310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:42:42.775916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:42:42.776196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:42:42.776474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:42:42.785236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:42:42.786810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba10e21-745e-415a-a2cd-23cf2487b162 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:42:42.797282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1217/2430] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:42:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:42:52.792344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:43:04.228202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:43:04.236723 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a455da-cf8b-4666-8ae4-8b705367c7d2 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:43:04.240340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:43:04.240949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:43:04.241262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:43:04.241546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:43:04.243349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] Could not find project: 4d888dcf554944adae3e03bc1c24c52b.: keystone.exception.ProjectNotFound: Could not find project: 4d888dcf554944adae3e03bc1c24c52b. Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 27 20:43:04.255044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:43:04.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 20:43:04.257668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 20:43:04.258827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 20:43:04.259916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4d888dcf554944adae3e03bc1c24c52b. Jul 27 20:43:04.259916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jul 27 20:43:04.259916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1214/2431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:43:04 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 29 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 27 20:43:14.246894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:43:21.941416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:43:21.941938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:43:21.942080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:43:21.970847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] MFA Rules not processed for user `b734a345bfe5483d9a5a1e35d89c935b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:43:21.972087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:43:21.979032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1218/2432] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:43:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:43:31.968445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:04.437483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:04.446542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892af8b2-ca1f-4c22-a044-604f4f07f231 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:04.450121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:04.450715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:04.450996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:04.451270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:04.459880 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:04.461372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:04.470657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1215/2433] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:04.644867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:04.654548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7852e-4db5-44fe-aed8-00dfd8b0276f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:04.658777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:04.659333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:04.659623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:04.659899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:04.668278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:04.669741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:04.678921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1219/2434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:12.566125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:12.574859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1926911-5b91-416b-a620-6e9c459c4648 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:12.579152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:12.579756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:12.580038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:12.580315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:12.590925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:12.592589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:12.604183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1216/2435] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:13.690030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:13.700021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433b2d-a1ec-46cf-97c9-4e0df7d89992 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:13.705963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:13.706584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:13.706870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:13.707149 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:13.708929 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:13.802289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:13.803808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:13.822715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1220/2436] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:14.291983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:14.293588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:14.376980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8430a3-b88f-449b-9803-c763ead44b0f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:14.380660 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:14.381306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:14.381591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:14.381865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:14.392446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:14.393897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:14.404608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1217/2437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:15.255250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:15.270169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b619d93-df5b-495a-9b2e-edafe00fd2d9 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:15.277088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:15.277703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:15.277979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:15.278256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:15.291573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:15.293706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:15.304691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1221/2438] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:24.399876 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=47 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:25.300056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=55 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:42.695144 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:42.705386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1203a55-5f65-42b4-bef4-eadcf3267c02 service glance] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:42.709020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:42.709632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:42.709916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:42.710196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:42.717494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:42.718926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:42.726962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1218/2439] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:45.485672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:45.495367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c048-54f9-4dd7-a921-e88bb7f1a1ba service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:45.499075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:45.499682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:45.499964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:45.500786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:45.502625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:45.514718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:45.516783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:45.527042 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1222/2440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:48.948107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:48.960639 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e3d5f-b7d9-431f-b7fa-d78007b39275 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:48.965851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:48.966564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:48.966834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:48.967112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:48.968878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:48.980697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:48.982338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.002829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.003382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1219/2441] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:49.005133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:49.019322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28536aa-3e6e-4bc9-a9d5-c24b6b074ba9 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.024153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.024787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.025100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.025377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] PATH_INFO: `/v3/users/ea3c5c3007e04391b5b108473da62787` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.026535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3c5c3007e04391b5b108473da62787)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.028446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.076510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3c5c3007e04391b5b108473da62787'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.090276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3c5c3007e04391b5b108473da62787'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.091089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3c5c3007e04391b5b108473da62787'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.098185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3c5c3007e04391b5b108473da62787'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.105975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-92e83974-4958-4973-95dd-052a718ced93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3c5c3007e04391b5b108473da62787'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.112505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1223/2442] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:45:49 2022] DELETE /identity/v3/users/ea3c5c3007e04391b5b108473da62787 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:45:49.285287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.296569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc16c0-9e93-4f36-9766-83bc5a00f6eb service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.361114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.361759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.362074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.362349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] PATH_INFO: `/v3/projects/8d9a9e3a2b0e4918950cfd454aab9fbb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.363514 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9a9e3a2b0e4918950cfd454aab9fbb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.365472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.413917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9a9e3a2b0e4918950cfd454aab9fbb', 'request_id': 'req-933c8b48-f743-4355-8070-479c5bedf72e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.419540 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9a9e3a2b0e4918950cfd454aab9fbb', 'request_id': 'req-933c8b48-f743-4355-8070-479c5bedf72e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.426012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9a9e3a2b0e4918950cfd454aab9fbb', 'request_id': 'req-933c8b48-f743-4355-8070-479c5bedf72e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.441179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] The token cache is being invalidate because project 8d9a9e3a2b0e4918950cfd454aab9fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:45:49.441475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.442110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-933c8b48-f743-4355-8070-479c5bedf72e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:45:49.443631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1220/2443] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:45:49 2022] DELETE /identity/v3/projects/8d9a9e3a2b0e4918950cfd454aab9fbb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:45:49.448843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.449214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.449653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.510178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:49.511424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:49.568789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1224/2444] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:49.576621 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.576908 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.577481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.597279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:49.598469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:49.606205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1221/2445] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:49.611226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.620887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85a38e-9cd7-43f7-ae35-30dc012fe888 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.626075 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.626756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.627034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.627303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.627988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:45:49.628292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.629934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.641434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1225/2446] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:45:49.645809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.655062 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6fac09-8998-4337-85ed-7fbc761de2e1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.659954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.660571 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.660853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.661155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.661833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.663746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.700592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1222/2447] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:45:49 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:45:49.704987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.715375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3519d-8f76-4dac-b1f0-9025e68104d8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.726980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:49.775671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.776296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.776580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.776858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.777550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.779475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.802914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1226/2448] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:45:49 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:45:49.807052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.818552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d817d3e-d782-4062-b0d6-92a728070864 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.823299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.823918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.824205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.824484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.825220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.826831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.837764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1223/2449] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:45:49.841568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.852180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bb68-a5c4-44c3-9ee1-5108534e5209 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.856865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.857507 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.857790 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.858066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.858762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.860941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3023d6-06c9-436a-b3b3-68cf79db6c74 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.876234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1227/2450] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:45:49.877715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:49.890755 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba4053-d92c-4bc8-97fe-92ea88243d1c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:49.896408 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:49.897275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.897623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.897972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] PATH_INFO: `/v3/projects/6a1fbeedf5134a5a9432e42a35e1d955/users/30051341684845938161bb5ec0680d68/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.911044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:49.915309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1fbeedf5134a5a9432e42a35e1d955, user_id=30051341684845938161bb5ec0680d68, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:49.918712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:49.934182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1224/2451] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:45:49 2022] PUT /identity/v3/projects/6a1fbeedf5134a5a9432e42a35e1d955/users/30051341684845938161bb5ec0680d68/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:45:49.941201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e70-54dc-40f9-a9b0-f47597ca76fb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:49.941496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e70-54dc-40f9-a9b0-f47597ca76fb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:49.941982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439e70-54dc-40f9-a9b0-f47597ca76fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:49.971011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e8439e70-54dc-40f9-a9b0-f47597ca76fb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:49.972608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e8439e70-54dc-40f9-a9b0-f47597ca76fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:50.041518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1228/2452] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:45:50.049211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:50.051014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:50.133735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cf530-6abc-4c4d-9016-b462b794fb01 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:50.137419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:50.138045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:50.138327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:50.138600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:50.149150 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:50.150593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:50.165697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1225/2453] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:51.547440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:51.548439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:51.548740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:51.576798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:51.579342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:51.644670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1229/2454] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:45:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:51.703101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:51.705378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:51.848684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda60682-dde0-4dad-a34a-5e573b1cda3f service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:51.863932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:51.864918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:51.865263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:51.865545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:51.883756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:51.885426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db878-9f0f-4cab-bf8e-0fd390a48775 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:51.898922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1226/2455] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:45:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:45:52.039886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:52.041989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:52.148430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:52.239012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d91c0-c157-43c3-90d3-70985d0b5fe9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:52.248685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:52.249624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:52.250449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:52.253707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:52.258016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:52.279592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:52.281896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:52.295710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1230/2456] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:54.993591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056b98-d0e4-463a-bd85-4f95758d9334 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:54.994079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056b98-d0e4-463a-bd85-4f95758d9334 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:54.994183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056b98-d0e4-463a-bd85-4f95758d9334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:55.049454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6c056b98-d0e4-463a-bd85-4f95758d9334 None None] MFA Rules not processed for user `30051341684845938161bb5ec0680d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:55.051024 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6c056b98-d0e4-463a-bd85-4f95758d9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:55.173019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1227/2457] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:45:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:55.189784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:55.191802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:55.207833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb808d3-2ad5-4831-bf25-192a209b4264 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:55.229388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:55.231151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:55.231442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:55.231735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:55.248272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:55.251274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:55.262199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1231/2458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:57.243974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:57.244489 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:57.244596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:57.259391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:45:57.276613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] MFA Rules not processed for user `30051341684845938161bb5ec0680d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:45:57.277893 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:57.290436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1228/2459] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:45:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:45:57.686847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:57.692661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:57.707939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4735b-8eeb-4c50-8f7e-5d80cb9c40a1 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:57.711832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:57.712449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:57.712952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:57.713403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:57.715445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:57.727980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:57.729659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:57.741514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1232/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:45:58.321637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:45:58.323066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:58.507043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd529-dcb9-4116-bd56-8a9d44306b5a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:45:58.527822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:45:58.528731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:45:58.529390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:45:58.529851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:45:58.532450 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:45:58.554435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:45:58.557241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:45:58.569633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1229/2461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:45:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:07.735601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:08.565281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=34 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:10.508514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:10.510881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:10.571155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d3a38-d2da-4291-a243-fdcf83fe5f55 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:10.574809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:10.575429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:10.575720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:10.575994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:10.578655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:10.592390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:10.593934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:10.605294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1233/2462] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:10.675917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:10.685297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefdb36-e483-4ff3-ba14-4e9ff64341f0 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:10.689702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:10.690306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:10.690585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:10.690857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:10.692581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:10.702699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:10.704244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742b0d3-361c-4e2b-a350-1ed9870931dd service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:10.711962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1230/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:46:10.794766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:10.804671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c8914-2c6d-4741-b351-fe3e4be0f87a service glance] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:10.808244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:10.808838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:10.809147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:10.809422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:10.811136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:10.825226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:10.826639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:10.839256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1234/2464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:46:13.573388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c0d4-4724-4cb3-9ff6-051197a4c107 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:13.573929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c0d4-4724-4cb3-9ff6-051197a4c107 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:13.574133 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c0d4-4724-4cb3-9ff6-051197a4c107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:13.597744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f109c0d4-4724-4cb3-9ff6-051197a4c107 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:46:13.599105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f109c0d4-4724-4cb3-9ff6-051197a4c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:13.608327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1231/2465] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:46:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:46:13.621836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:13.633777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf66ee-f802-42a2-aebc-0ebb6ec4904f service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:13.638818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:13.639493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:13.639884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:13.640216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:13.648576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:13.650260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:13.658685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1235/2466] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:46:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:46:23.591350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=9 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:23.655023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=38 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:40.462440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0821d-162e-4fe9-a376-b3f173854400 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:40.462914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0821d-162e-4fe9-a376-b3f173854400 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:40.463012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0821d-162e-4fe9-a376-b3f173854400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:40.481932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cea0821d-162e-4fe9-a376-b3f173854400 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:46:40.483128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cea0821d-162e-4fe9-a376-b3f173854400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:46:40.490896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1232/2467] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:46:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:46:40.504110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:46:40.514732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa09b-4f95-456f-acab-020d04437806 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:46:40.520067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:46:40.520672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:46:40.520954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:46:40.521282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:46:40.529376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:46:40.530948 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:46:40.539432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1236/2468] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:46:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:46:50.476580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:46:50.536099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:41.569151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:41.569636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:41.569758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:41.595137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] MFA Rules not processed for user `30051341684845938161bb5ec0680d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:41.596379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:41.621724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1233/2469] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:42.998193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:43.051107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5d9f-207d-4040-a431-1c417e968e11 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:43.056287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:43.056904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:43.057223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:43.057503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:43.059267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:43.083855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:43.085516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:43.095889 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1237/2470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:44.808223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:44.820830 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1f2aa-de31-4229-b0df-d03bd9ff24fc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:44.825243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:44.826087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:44.826560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:44.827273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:44.829335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:44.847436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:44.854309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:44.872734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1234/2471] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:49.036233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.038302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:49.060292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018194d-35c9-4ac1-8266-14071eeedb8d service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.064843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.065632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.066675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.066996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] PATH_INFO: `/v3/users/b734a345bfe5483d9a5a1e35d89c935b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.082859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b734a345bfe5483d9a5a1e35d89c935b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.085070 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.133703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b734a345bfe5483d9a5a1e35d89c935b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.145330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b734a345bfe5483d9a5a1e35d89c935b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.145985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b734a345bfe5483d9a5a1e35d89c935b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.152599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b734a345bfe5483d9a5a1e35d89c935b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.160458 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8b3b50d7-0612-44d9-bdc8-d0516d58daa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b734a345bfe5483d9a5a1e35d89c935b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.166336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1238/2472] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:47:49 2022] DELETE /identity/v3/users/b734a345bfe5483d9a5a1e35d89c935b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:49.336886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.349486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df57ddc-8adb-4513-ad54-26986e85b316 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.407088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.407733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.408019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.408304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] PATH_INFO: `/v3/projects/ec01b599458b451e8d04a428c1c33513` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.410258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec01b599458b451e8d04a428c1c33513)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.412707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.462427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec01b599458b451e8d04a428c1c33513', 'request_id': 'req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.468385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec01b599458b451e8d04a428c1c33513', 'request_id': 'req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.475129 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec01b599458b451e8d04a428c1c33513', 'request_id': 'req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.486901 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] The token cache is being invalidate because project ec01b599458b451e8d04a428c1c33513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:47:49.487188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.487848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3ea061cf-d79f-44d1-b5a6-36d9b113cad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:49.489428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1235/2473] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:47:49 2022] DELETE /identity/v3/projects/ec01b599458b451e8d04a428c1c33513 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:49.494400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.494693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.494978 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.514252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:49.515481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:49.573859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1239/2474] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:49.582651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7a56-25d9-4183-9862-51059b44354b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.583004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7a56-25d9-4183-9862-51059b44354b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.583281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c7a56-25d9-4183-9862-51059b44354b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.603478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-355c7a56-25d9-4183-9862-51059b44354b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:49.604701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-355c7a56-25d9-4183-9862-51059b44354b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:49.613438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1236/2475] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:49.617570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.622527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:49.629419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9248f4a-3022-416b-a7ae-495117998ea3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.635397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.636016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.636302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.636578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.637314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:47:49.637636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.639529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.652402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1240/2476] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:47:49.656758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7a56-25d9-4183-9862-51059b44354b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.666625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c7a56-25d9-4183-9862-51059b44354b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.671223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.671827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.672103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.672379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.673192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.675131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.706608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1237/2477] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:49 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:49.708094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.718713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2343b6-42cb-4a1f-b7e9-722e57ebff08 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.777024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.777648 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.777930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.778207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.778878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.780798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.801424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1241/2478] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:49 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:49.805789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.816466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a98dd-3785-4296-b093-58b6a6363ac2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.821564 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.822246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.822533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.822818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.823515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.825166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.836169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1238/2479] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:49.840025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.850379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6a54-e5d1-4ab3-a8e1-2ca72494c559 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.855031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.855633 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.855917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.856197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.856887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.858527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f31e-be78-4251-aa7d-cdd532154dbe admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.868939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1242/2480] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:49.872998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:49.883615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b97fa-6415-4e9b-b4dd-7f2e712e0741 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:49.889613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:49.890789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.891319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.891839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] PATH_INFO: `/v3/projects/4ab10a5809ed4b3eab991d540b6e5ab0/users/1982c1753d544a89bdee471d329aefde/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.907475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab10a5809ed4b3eab991d540b6e5ab0, user_id=1982c1753d544a89bdee471d329aefde, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:49.910246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:49.912418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:49.928769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1239/2481] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:49 2022] PUT /identity/v3/projects/4ab10a5809ed4b3eab991d540b6e5ab0/users/1982c1753d544a89bdee471d329aefde/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:49.941214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:49.941574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:49.941985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:49.970769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] MFA Rules not processed for user `1982c1753d544a89bdee471d329aefde`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:49.972415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:50.038959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1243/2482] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:50.051143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:50.052905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:50.136076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15d43d-11fc-420c-962c-1cef4df6ed80 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:50.141594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:50.142207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:50.142485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:50.142757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:50.150352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:50.151924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547b878-6b06-4d97-b9b7-1adf0f2f62ed service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:50.161007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1240/2483] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:52.850234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:52.908481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:52.913246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037dcb-b5c9-4351-becd-aca19b0d4ac8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:52.918953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:52.919592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:52.919885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:52.920165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] PATH_INFO: `/v3/users/1982c1753d544a89bdee471d329aefde` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:52.921384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1982c1753d544a89bdee471d329aefde)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:52.923320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:52.968719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1982c1753d544a89bdee471d329aefde'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:52.980314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1982c1753d544a89bdee471d329aefde'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:52.980985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1982c1753d544a89bdee471d329aefde'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:52.987829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1982c1753d544a89bdee471d329aefde'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:52.995072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1982c1753d544a89bdee471d329aefde'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.001713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1244/2484] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:47:52 2022] DELETE /identity/v3/users/1982c1753d544a89bdee471d329aefde => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:53.006463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.006971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.007084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.027891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:53.029148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.102827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1241/2485] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:53.114108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.115768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.188281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669de820-1aa3-44a3-9144-54b2bc4d193d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.192485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.193158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.193555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.193834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.203166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.204730 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14832a-8119-409e-a41e-bf6fb8c66342 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.215122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1245/2486] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:47:53.535069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.536520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.548500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed775c33-5400-46f1-afcc-f0c05a686e7b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.553797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.554442 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.554740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.555002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] PATH_INFO: `/v3/projects/4ab10a5809ed4b3eab991d540b6e5ab0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.556160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab10a5809ed4b3eab991d540b6e5ab0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.558110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.606846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab10a5809ed4b3eab991d540b6e5ab0', 'request_id': 'req-768fdb58-19b5-4805-a095-6a2d0954681b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.613075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab10a5809ed4b3eab991d540b6e5ab0', 'request_id': 'req-768fdb58-19b5-4805-a095-6a2d0954681b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.619278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab10a5809ed4b3eab991d540b6e5ab0', 'request_id': 'req-768fdb58-19b5-4805-a095-6a2d0954681b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.631756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] The token cache is being invalidate because project 4ab10a5809ed4b3eab991d540b6e5ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:47:53.632040 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.632648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-768fdb58-19b5-4805-a095-6a2d0954681b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:53.634172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1242/2487] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:47:53 2022] DELETE /identity/v3/projects/4ab10a5809ed4b3eab991d540b6e5ab0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:47:53.638996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.639287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.639568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.658819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:53.660756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.717622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1246/2488] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:53.725751 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.726036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.726306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.734164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:53.748547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:53.749789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:53.757656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1243/2489] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:53.761754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.771406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6e275-3ae9-4f31-b99c-f9045b0f3186 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.776054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.776685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.776994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.777274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.777962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:47:53.778268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.779859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.791708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1247/2490] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:47:53.795789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.805466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45def4-ec37-4f2b-ab22-7654ec5454a4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.811009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.811607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.811885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.812179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.812834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.814787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.841298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1244/2491] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:53 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:53.845390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.855839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bea658-6d10-4f37-b5bb-4ef4e7e7392b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.859402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:53.919358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.919977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.920258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.920535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.921235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.923208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.943946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1248/2492] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:53 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:53.948086 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.959052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c5bcf-8441-48d0-9859-bf8a9db7e6ab admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.963649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.964362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.964648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.964950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.965665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:53.967237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:53.977827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1245/2493] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:47:53.981737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:53.992232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8eb530-197d-4e34-a46e-dedacbc15b57 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:53.996854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:53.997485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:53.997766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:53.998037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:53.998723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.000299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8907ca8-d7e6-47b1-a886-853c5e402e5b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.011667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1249/2494] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:54.015665 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.026083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500af3f-7934-4787-837e-bff9cb9d68ba admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.030798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.031424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.031737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.032014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] PATH_INFO: `/v3/projects/99cce27d2e974e90b6fd9ec4e6a67fd4/users/ee9f68e41767466c95578861312d72ab/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.046409 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] RBAC: Authorizing `identity:create_grant(project_id=99cce27d2e974e90b6fd9ec4e6a67fd4, user_id=ee9f68e41767466c95578861312d72ab, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.049182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.062244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1246/2495] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:54 2022] PUT /identity/v3/projects/99cce27d2e974e90b6fd9ec4e6a67fd4/users/ee9f68e41767466c95578861312d72ab/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:47:54.072847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.073176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.073453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.101772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] MFA Rules not processed for user `ee9f68e41767466c95578861312d72ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:54.102966 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:54.165237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1250/2496] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:54.169757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.232885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62442dd9-4f31-4dc4-835e-eab73ebd01be admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.237823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.238439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.238722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.239002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.239676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.241628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.267981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1247/2497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:54 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:54.272656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.283168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59084-5164-4421-a1c0-3eb8d19a0e3e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.337588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.338371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.340027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.340344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.341082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.343041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.368230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1251/2498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:54 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:54.372417 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.383038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175452f-7af6-499a-958a-d3ef7dda9d61 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.388118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.388851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.389244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.389634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.390381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.392121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.403389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1248/2499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:54.408150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.416452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:54.420516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb20787-5437-44ae-b150-0857ff45f98e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.427698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.428422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.428995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.429283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] PATH_INFO: `/v3/projects/90b90b8285db4fb7a439449028b9178b/users/2248fb4ab4304a0bb5bc3551298590b5/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.446781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b90b8285db4fb7a439449028b9178b, user_id=2248fb4ab4304a0bb5bc3551298590b5, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.449568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.464785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1252/2500] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:54 2022] PUT /identity/v3/projects/90b90b8285db4fb7a439449028b9178b/users/2248fb4ab4304a0bb5bc3551298590b5/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:54.468829 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.479727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbe621-d676-49f9-8a68-711bf52fe0b1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.542371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.543000 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.543907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.544197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.544904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.546538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30847e7-241e-44d7-95c0-277f4557a393 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.558732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1249/2501] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:54.563336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.575535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855597f8-a660-4e20-8bfe-a09cf5c756d3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.580825 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.581549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.582327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.582327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] PATH_INFO: `/v3/projects/90b90b8285db4fb7a439449028b9178b/users/2248fb4ab4304a0bb5bc3551298590b5/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.584248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b90b8285db4fb7a439449028b9178b, user_id=2248fb4ab4304a0bb5bc3551298590b5, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.586937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.598316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1253/2502] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:54 2022] PUT /identity/v3/projects/90b90b8285db4fb7a439449028b9178b/users/2248fb4ab4304a0bb5bc3551298590b5/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:47:54.610054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.610370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.610649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.634444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:54.646016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] MFA Rules not processed for user `2248fb4ab4304a0bb5bc3551298590b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:54.647562 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:54.719456 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1250/2503] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:54.730534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:54.732669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:54.792273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598b85-521f-4981-b88c-57b9fe24ab21 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:54.795957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:54.796600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:54.796877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:54.797179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:54.807182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:54.808612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba629e-e4cb-4999-a0be-b179aeb23811 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:54.818431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1254/2504] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:54.969525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:55.036211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fa6f5-c87d-48ab-8e0d-d1c8f0aec141 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:55.041008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:55.041671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:55.041969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:55.042234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] PATH_INFO: `/v3/users/ee9f68e41767466c95578861312d72ab` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:55.043459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9f68e41767466c95578861312d72ab)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:55.045497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:55.101767 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9f68e41767466c95578861312d72ab'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.109160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9f68e41767466c95578861312d72ab'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.121568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9f68e41767466c95578861312d72ab'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.129613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9f68e41767466c95578861312d72ab'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.130288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9f68e41767466c95578861312d72ab'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.136163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1251/2505] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:47:54 2022] DELETE /identity/v3/users/ee9f68e41767466c95578861312d72ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:55.140752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:55.141257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:55.141384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:55.160477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:55.161743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:55.223817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1255/2506] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:55.233627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:55.235067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:55.250311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:55.295679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29e53d-812e-4ea1-b1b5-ba24cd3913ff admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:55.299253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:55.300602 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:55.300986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:55.301275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:55.310197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:55.311614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:55.320949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1252/2507] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:55.589301 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:55.590926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:55.604383 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149f1fd-1324-4c96-a7c3-bc98987b0712 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:55.609227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:55.609958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:55.610375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:55.610757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] PATH_INFO: `/v3/projects/99cce27d2e974e90b6fd9ec4e6a67fd4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:55.612019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=99cce27d2e974e90b6fd9ec4e6a67fd4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:55.614292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:55.663800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99cce27d2e974e90b6fd9ec4e6a67fd4', 'request_id': 'req-714b92e6-5fcb-41bd-a752-093899218ebc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.669141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99cce27d2e974e90b6fd9ec4e6a67fd4', 'request_id': 'req-714b92e6-5fcb-41bd-a752-093899218ebc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.675652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99cce27d2e974e90b6fd9ec4e6a67fd4', 'request_id': 'req-714b92e6-5fcb-41bd-a752-093899218ebc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.678243 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:55.690693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] The token cache is being invalidate because project 99cce27d2e974e90b6fd9ec4e6a67fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:47:55.690987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.691719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.693429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1256/2508] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:47:55 2022] DELETE /identity/v3/projects/99cce27d2e974e90b6fd9ec4e6a67fd4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:55.697601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:55.708096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb78b-bc10-4775-891f-dc6ba3c12b0c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:55.764867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:55.765536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:55.765824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:55.766102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] PATH_INFO: `/v3/users/2248fb4ab4304a0bb5bc3551298590b5` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:55.767263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2248fb4ab4304a0bb5bc3551298590b5)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:55.769937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:55.815433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2248fb4ab4304a0bb5bc3551298590b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.821715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2248fb4ab4304a0bb5bc3551298590b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.835481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2248fb4ab4304a0bb5bc3551298590b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.842179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2248fb4ab4304a0bb5bc3551298590b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.842826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-30a4723b-99d4-4491-8572-4271329b91d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2248fb4ab4304a0bb5bc3551298590b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:55.849578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1253/2509] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:47:55 2022] DELETE /identity/v3/users/2248fb4ab4304a0bb5bc3551298590b5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:56.130303 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.131753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:56.145363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b92e6-5fcb-41bd-a752-093899218ebc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.202395 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.203034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.203313 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.203588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] PATH_INFO: `/v3/projects/90b90b8285db4fb7a439449028b9178b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.204740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b90b8285db4fb7a439449028b9178b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.207305 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.261583 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b90b8285db4fb7a439449028b9178b', 'request_id': 'req-cbab04e4-9259-45ec-9a12-ddb1d30172e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:56.267233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b90b8285db4fb7a439449028b9178b', 'request_id': 'req-cbab04e4-9259-45ec-9a12-ddb1d30172e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:56.274296 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b90b8285db4fb7a439449028b9178b', 'request_id': 'req-cbab04e4-9259-45ec-9a12-ddb1d30172e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:56.286539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] The token cache is being invalidate because project 90b90b8285db4fb7a439449028b9178b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:47:56.286829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:56.287450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cbab04e4-9259-45ec-9a12-ddb1d30172e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:47:56.288956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1257/2510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:47:56 2022] DELETE /identity/v3/projects/90b90b8285db4fb7a439449028b9178b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:56.293973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.294445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.294567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.314401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:56.316223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:56.351858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:56.383894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1254/2511] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:56.391643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.391928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.392204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.411723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:56.412925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:56.420831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1258/2512] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:56.425612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.435436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97eea1-08a2-45aa-aeb1-c3d9bb1ea424 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.440120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.441588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.441872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.442147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.442835 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:47:56.443141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.444738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.457278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1255/2513] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:47:56.462770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.476551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c2e28-d148-46dc-b15b-c8280d072d11 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.481403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.482013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.482291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.482562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.483235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.485200 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.513025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1259/2514] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:56.517366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.527762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8019ea-24bd-4e05-b91c-0cb453f48d85 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.584649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.585212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.585498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.585773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.586437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.588349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.610737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1256/2515] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:56.615851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.626112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d207-7dd8-4dfa-8010-b2abf8fd543e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.631251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.631890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.632172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.632454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.633185 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.634795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.645071 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1260/2516] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:56.649064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.658975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575cb71b-c816-45f0-9e80-7b36b51d462f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.664617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.665307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.665594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.665873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.666577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.668193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ae7fc-2f57-48c3-bff5-8bbccea2324d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.680467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1257/2517] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:56.684745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.695060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bc31b-b060-46a4-9d63-d430f1f384a0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.699751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.700408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.700692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.701033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] PATH_INFO: `/v3/projects/74c01fbf161a43c5a1a77083fa56f8a6/users/632b42e2c6dc4464bbca38a458194675/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.715680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c01fbf161a43c5a1a77083fa56f8a6, user_id=632b42e2c6dc4464bbca38a458194675, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.719734 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.734317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1261/2518] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:56 2022] PUT /identity/v3/projects/74c01fbf161a43c5a1a77083fa56f8a6/users/632b42e2c6dc4464bbca38a458194675/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:56.745975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.746269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.746551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.774060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] MFA Rules not processed for user `632b42e2c6dc4464bbca38a458194675`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:56.775280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:56.844707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1258/2519] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:56.845792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.868099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:56.912816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab391e-d0fd-4dfb-8f0e-f225340e90dc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.917578 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:56.918197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:56.918482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:56.918761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:56.919439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:56.921407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:56.952828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1262/2520] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:56.954070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:56.965134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81751094-b31b-49a5-88e1-a78b35db19fa None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:56.995051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:57.024595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.025250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.025538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.025838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.026483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.028422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.050172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1259/2521] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:56 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:57.054729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.066556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df71e0-1a4b-4839-93c1-72cd248d44c9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.071618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.072242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.072524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.072798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.073539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.075308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.087056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1263/2522] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:57.091079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.101821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc7386-c853-4721-8f6f-b8234c709cae admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.106464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.107078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.107366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.107641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.108341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.109972 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bb79-295f-424d-95a4-126715f58378 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.122458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1260/2523] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:47:57.126663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.137935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fbdc4-936d-4cfa-af85-bb3d3772eb01 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.142785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.144324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.144630 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.144921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] PATH_INFO: `/v3/projects/d19065fd35f34d51b7b92e0867ca4a39/users/ea6be884717b4ba0bc66089f5db1bf87/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.159738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19065fd35f34d51b7b92e0867ca4a39, user_id=ea6be884717b4ba0bc66089f5db1bf87, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.162465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.177586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1264/2524] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:57 2022] PUT /identity/v3/projects/d19065fd35f34d51b7b92e0867ca4a39/users/ea6be884717b4ba0bc66089f5db1bf87/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:57.188875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.189294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.189600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.220318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] MFA Rules not processed for user `ea6be884717b4ba0bc66089f5db1bf87`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:57.222267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:57.289679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1261/2525] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:57.294116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.361612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e218e61-2a0a-4736-b7de-a9b890f7b445 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.367240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.368238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.368586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.369029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.369881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.371851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.415227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1265/2526] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:47:57 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:47:57.416338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.427177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f443313-5167-415e-8bf5-41153ae0a36a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.617285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.617913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.618196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.618472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.619134 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.621100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.644390 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1262/2527] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:47:57 2022] POST /identity/v3/users => generated 295 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:47:57.648545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.658679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159d36-493a-4675-b415-03fb5cb22536 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.663844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.664448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.664723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.665016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.665757 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.667360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.678735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1266/2528] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:57.683488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.694649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b709d-d4c4-4290-8409-fca3c45b9031 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.699369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.700034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.700321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.700600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] PATH_INFO: `/v3/projects/b394090a67074fac9689822306be761c/users/6b3f3f26a9904a5b9e2d6c50a858257e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.716824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b394090a67074fac9689822306be761c, user_id=6b3f3f26a9904a5b9e2d6c50a858257e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.719616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.733494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1263/2529] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:57 2022] PUT /identity/v3/projects/b394090a67074fac9689822306be761c/users/6b3f3f26a9904a5b9e2d6c50a858257e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:47:57.738342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.748506 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927ef55-abca-40e7-b95c-97e285d31105 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.796252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:57.814344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.815335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.815816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.816161 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.817013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.818638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7efb44-9f4a-4960-ba97-d2c40ecf82be admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.828922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1267/2530] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:47:57.834211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:57.841634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:47:57.846135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a52d80-03aa-4ff6-9be4-5a034dc94b7a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:57.856865 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:57.857840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.858181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.858563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] PATH_INFO: `/v3/projects/b394090a67074fac9689822306be761c/users/6b3f3f26a9904a5b9e2d6c50a858257e/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.860853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b394090a67074fac9689822306be761c, user_id=6b3f3f26a9904a5b9e2d6c50a858257e, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:57.864174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:57.875342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1264/2531] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:47:57 2022] PUT /identity/v3/projects/b394090a67074fac9689822306be761c/users/6b3f3f26a9904a5b9e2d6c50a858257e/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:47:57.885749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:57.886041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:57.886330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:57.915851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] MFA Rules not processed for user `6b3f3f26a9904a5b9e2d6c50a858257e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:47:57.917209 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:57.998347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1268/2532] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:47:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:47:58.010800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:58.012725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:58.079735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6481c9-a527-4a6b-8b81-fd9823691b0d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:58.085160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:58.085909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:58.086258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:58.086774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:58.101437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:58.102882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:58.114617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1265/2533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:47:58.161442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:47:58.162972 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:47:58.175947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a57ec4-5ee7-4e80-9bad-e045a1210481 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:47:58.186225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:47:58.187067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:47:58.187459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:47:58.187805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:47:58.259763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:47:58.261585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd7962-d0c8-4ad0-9c0e-e5172941987c service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:47:58.272012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1269/2534] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:47:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:01.216314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:01.276867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dfe2e-afe4-4572-8ae8-6bec1312ab47 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:01.281267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:01.281900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:01.282174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:01.282444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] PATH_INFO: `/v3/users/632b42e2c6dc4464bbca38a458194675` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:01.283612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=632b42e2c6dc4464bbca38a458194675)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:01.286182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:01.330191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632b42e2c6dc4464bbca38a458194675'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.337204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632b42e2c6dc4464bbca38a458194675'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.349389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632b42e2c6dc4464bbca38a458194675'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.356070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632b42e2c6dc4464bbca38a458194675'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.356699 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632b42e2c6dc4464bbca38a458194675'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.363076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1266/2535] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:01 2022] DELETE /identity/v3/users/632b42e2c6dc4464bbca38a458194675 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:01.367477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:01.367956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:01.368076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:01.386492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:01.387696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:01.444725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1270/2536] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:01.454659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:01.456606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:01.472760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:01.516726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cad51-ee33-433b-b1ac-85c181d30f75 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:01.520864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:01.521500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:01.521782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:01.522053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:01.530816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:01.532199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:01.541909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1267/2537] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:01.815036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:01.816613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:01.825468 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:01.834430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e59a03-68c1-444e-9380-a28845bf4e4e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:01.839394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:01.840043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:01.840322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:01.840594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] PATH_INFO: `/v3/projects/74c01fbf161a43c5a1a77083fa56f8a6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:01.842053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c01fbf161a43c5a1a77083fa56f8a6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:01.843879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:01.889245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c01fbf161a43c5a1a77083fa56f8a6', 'request_id': 'req-3d2f7652-5b80-48da-9694-38229cad9b61'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.895288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c01fbf161a43c5a1a77083fa56f8a6', 'request_id': 'req-3d2f7652-5b80-48da-9694-38229cad9b61'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.901372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c01fbf161a43c5a1a77083fa56f8a6', 'request_id': 'req-3d2f7652-5b80-48da-9694-38229cad9b61'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.913457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] The token cache is being invalidate because project 74c01fbf161a43c5a1a77083fa56f8a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:01.913744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.914356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:01.915840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1271/2538] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:01 2022] DELETE /identity/v3/projects/74c01fbf161a43c5a1a77083fa56f8a6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:01.919832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:01.929653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37e72f-881d-48e0-b4f3-9e666f95eebb service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:01.987223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:01.987859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:01.988143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:01.988421 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] PATH_INFO: `/v3/users/ea6be884717b4ba0bc66089f5db1bf87` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:01.989580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6be884717b4ba0bc66089f5db1bf87)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:01.991669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:02.037586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6be884717b4ba0bc66089f5db1bf87'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.044902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6be884717b4ba0bc66089f5db1bf87'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.058854 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6be884717b4ba0bc66089f5db1bf87'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.066244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6be884717b4ba0bc66089f5db1bf87'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.066950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6be884717b4ba0bc66089f5db1bf87'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.072978 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1268/2539] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:01 2022] DELETE /identity/v3/users/ea6be884717b4ba0bc66089f5db1bf87 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:02.344281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:02.346068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:02.359545 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f7652-5b80-48da-9694-38229cad9b61 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:02.415189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:02.415878 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:02.416091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:02.416363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] PATH_INFO: `/v3/projects/d19065fd35f34d51b7b92e0867ca4a39` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:02.417531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19065fd35f34d51b7b92e0867ca4a39)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:02.419430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:02.465306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19065fd35f34d51b7b92e0867ca4a39', 'request_id': 'req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.470856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19065fd35f34d51b7b92e0867ca4a39', 'request_id': 'req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.477415 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19065fd35f34d51b7b92e0867ca4a39', 'request_id': 'req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.489856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] The token cache is being invalidate because project d19065fd35f34d51b7b92e0867ca4a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:02.490146 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.490766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.492244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1272/2540] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:02 2022] DELETE /identity/v3/projects/d19065fd35f34d51b7b92e0867ca4a39 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:02.496265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:02.497718 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:02.511269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06133de0-7443-411e-96b9-7a0d760a4159 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:02.543907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:02.570735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:02.571375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:02.571661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:02.571938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] PATH_INFO: `/v3/users/6b3f3f26a9904a5b9e2d6c50a858257e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:02.573122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3f3f26a9904a5b9e2d6c50a858257e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:02.575048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:02.620733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3f3f26a9904a5b9e2d6c50a858257e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.628988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3f3f26a9904a5b9e2d6c50a858257e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.642712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3f3f26a9904a5b9e2d6c50a858257e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.650029 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3f3f26a9904a5b9e2d6c50a858257e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.650806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4aa6f99a-035a-4360-9078-75b83c0d1c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3f3f26a9904a5b9e2d6c50a858257e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:02.656923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1269/2541] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:02 2022] DELETE /identity/v3/users/6b3f3f26a9904a5b9e2d6c50a858257e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:02.924934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:02.926404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:02.939328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22d3a-c490-4818-a3a6-aa29f6481e6f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:02.953293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:03.000510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.001194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.001591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.001876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] PATH_INFO: `/v3/projects/b394090a67074fac9689822306be761c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.003659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b394090a67074fac9689822306be761c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.005697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.050524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b394090a67074fac9689822306be761c', 'request_id': 'req-0c04c489-bf47-4238-9dc5-cb017716a3d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.057058 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b394090a67074fac9689822306be761c', 'request_id': 'req-0c04c489-bf47-4238-9dc5-cb017716a3d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.063120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b394090a67074fac9689822306be761c', 'request_id': 'req-0c04c489-bf47-4238-9dc5-cb017716a3d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.078613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] The token cache is being invalidate because project b394090a67074fac9689822306be761c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:03.078946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.079645 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c04c489-bf47-4238-9dc5-cb017716a3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.081624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1273/2542] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:02 2022] DELETE /identity/v3/projects/b394090a67074fac9689822306be761c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:03.087609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.087903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.088188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.107633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:03.108839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:03.169446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1270/2543] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:03.178026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.178310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.178582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.197815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:03.199001 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:03.207587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1274/2544] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:03.211915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.221482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdef3ef-e49d-4503-9418-6fad87678835 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.226739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.227343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.227627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.227904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.228593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:03.228899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.230515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.241841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1271/2545] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:48:03.246059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.256190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec92eb3-9838-4c6c-8890-51ab81185111 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.261754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.262370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.262654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.262932 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.263603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.265629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.293852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1275/2546] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:03.298206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.308529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d18b62-edfe-4b8d-b6c7-98b4defc84da admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.349746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:03.369092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.369703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.369982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.370255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.370949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.372850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.393293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1272/2547] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:03.398229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.408977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04230a-4b7f-45a7-87cf-701f852e3fbe admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.414338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.414944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.415231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.415798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.416187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.417829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.428590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1276/2548] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:03.432330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.442427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9335c3e-87d7-48d8-ba45-ab7e06fb5907 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.446959 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.447551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.447827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.448093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.448803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.450420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101afb7-02e3-4e44-9feb-bb63979e2a7e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.462192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1273/2549] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:03.466557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.477944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8bf63-02b1-40ae-af9f-47ed196bfc26 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.482828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.483484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.483770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.484051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] PATH_INFO: `/v3/projects/fa549d722da74e9bb63ae8e9e3410032/users/f13035a9076d4eb5b7f67827d1deafdf/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.499724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa549d722da74e9bb63ae8e9e3410032, user_id=f13035a9076d4eb5b7f67827d1deafdf, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.502451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.515080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1277/2550] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:03 2022] PUT /identity/v3/projects/fa549d722da74e9bb63ae8e9e3410032/users/f13035a9076d4eb5b7f67827d1deafdf/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:03.527846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.528364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.528364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.557737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] MFA Rules not processed for user `f13035a9076d4eb5b7f67827d1deafdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:03.558551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:03.622563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1274/2551] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:03.633824 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.635520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:03.682959 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:03.694479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cf882-10b3-4f11-9671-4c7c92e723f3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.698822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.699453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.700138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.700489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.710400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.711830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a864-227e-4fff-a4bb-122ba7ff7301 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.720901 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1278/2552] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:03.744245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.806385 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b2c0c-f507-45bf-8490-9142f60e1a5f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:03.811041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:03.811676 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.811960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.812238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] PATH_INFO: `/v3/users/f13035a9076d4eb5b7f67827d1deafdf` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.813441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13035a9076d4eb5b7f67827d1deafdf)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:03.815372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:03.860647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13035a9076d4eb5b7f67827d1deafdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.865102 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:03.869035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13035a9076d4eb5b7f67827d1deafdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.884064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13035a9076d4eb5b7f67827d1deafdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.890709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13035a9076d4eb5b7f67827d1deafdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.891361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13035a9076d4eb5b7f67827d1deafdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:03.897419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1275/2553] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:03 2022] DELETE /identity/v3/users/f13035a9076d4eb5b7f67827d1deafdf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:03.901933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:03.902222 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:03.902505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:03.924881 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:03.926911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:03.986050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1279/2554] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:48:03.996484 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:03.997955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:04.056761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b9833-0ec5-4076-9698-7f2b474f71f3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.060422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.061071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.061356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.061632 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.071492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.073269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d33cf-501e-445f-b97c-ee2931e54e3e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.083886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1276/2555] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:04.367215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.368738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:04.380789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523e68f-7017-4dbd-bc45-b07fcf659453 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.385968 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.386676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.387059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.387335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] PATH_INFO: `/v3/projects/fa549d722da74e9bb63ae8e9e3410032` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.388754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa549d722da74e9bb63ae8e9e3410032)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.390892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.438382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa549d722da74e9bb63ae8e9e3410032', 'request_id': 'req-d57f5906-293b-4d19-8ebf-95df56649edc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:04.443986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa549d722da74e9bb63ae8e9e3410032', 'request_id': 'req-d57f5906-293b-4d19-8ebf-95df56649edc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:04.450144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa549d722da74e9bb63ae8e9e3410032', 'request_id': 'req-d57f5906-293b-4d19-8ebf-95df56649edc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:04.462947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] The token cache is being invalidate because project fa549d722da74e9bb63ae8e9e3410032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:04.463235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:04.463852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d57f5906-293b-4d19-8ebf-95df56649edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:04.465374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1280/2556] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:04 2022] DELETE /identity/v3/projects/fa549d722da74e9bb63ae8e9e3410032 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:04.470366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.470654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.470930 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.491181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:04.492310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:04.549587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1277/2557] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:04.557887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.558180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.558461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.578802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:04.580014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:04.588128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1281/2558] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:04.592279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.601887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ca041-0a5a-408a-ad02-d273d602c28b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.606430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.607031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.607315 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.607594 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.608284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:04.608591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.610234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.622223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1278/2559] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:48:04.626316 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.636011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41237d8-1992-406d-85ea-b9cbb5a4c3f8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.640604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.641232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.641517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.641792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.642453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.644355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.671588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1282/2560] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:04.676428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.686539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907e870-082c-4ad3-96e7-595140154739 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.743966 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.744525 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.744806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.745155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.745815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.747709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.756454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:04.774335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1279/2561] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:04.775514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.786242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ede7a4-4059-4e71-8ce5-61c90b0d7fd1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.790832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.791429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.791735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.792011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.792698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.794330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.805182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1283/2562] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:04.809208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.819765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132ff6a-58d4-4ea1-8545-b33bf2de6bba admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.824517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.825143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.825427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.825700 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.826389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.828272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386efd3-9365-4c62-93ed-c4fb87540b57 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.839109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1280/2563] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:48:04.843514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:04.854342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aa0a5-8769-4914-9ad7-448c0d7eb567 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:04.859016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:04.859663 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.859970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.860246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] PATH_INFO: `/v3/projects/25af0bc01ae34b6a9ba63b517385b397/users/445fc365025c4406a65d8108c2e5e655/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.875163 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25af0bc01ae34b6a9ba63b517385b397, user_id=445fc365025c4406a65d8108c2e5e655, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:04.877877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:04.890310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1284/2564] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:04 2022] PUT /identity/v3/projects/25af0bc01ae34b6a9ba63b517385b397/users/445fc365025c4406a65d8108c2e5e655/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:48:04.901808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:04.902096 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:04.902397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:04.930033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] MFA Rules not processed for user `445fc365025c4406a65d8108c2e5e655`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:04.931239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:04.994770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1281/2565] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:04.999265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.007920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:05.060680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca241-3db2-4345-b8b7-c569e1313978 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.065726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.066387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.066746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.067102 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.067756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.069737 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.097210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1285/2566] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:04 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:05.101812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.112430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11886-6797-4585-9962-fb99e81a53a6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.168503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.169149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.169427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.169696 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.170350 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.172238 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.193731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1282/2567] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:05 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:05.197782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.208930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283f5d9-643b-495a-beaf-5700b4c93754 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.213612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.214213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.214498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.214777 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.215466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.217104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.228066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1286/2568] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:05.231803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.241738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26596942-7b11-46f3-a5a6-2489a9dcf182 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.246434 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.246957 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.247240 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.247517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.248207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.249832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f154158-967d-4d55-80c1-f9a7f6a5d6d9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.262050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1283/2569] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:05.269219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.278886 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb65a-cbb0-49e7-85ec-2a8be3c22e31 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.283626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.284278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.284559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.284838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] PATH_INFO: `/v3/projects/2f67bafdfcbb4b999cf95fc9b09a3f3b/users/6d80f2086b174a3294250e2bb308c364/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.299823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f67bafdfcbb4b999cf95fc9b09a3f3b, user_id=6d80f2086b174a3294250e2bb308c364, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.302896 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.315166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1287/2570] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:05 2022] PUT /identity/v3/projects/2f67bafdfcbb4b999cf95fc9b09a3f3b/users/6d80f2086b174a3294250e2bb308c364/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:05.325729 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.326104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.326410 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.355316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] MFA Rules not processed for user `6d80f2086b174a3294250e2bb308c364`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:05.356524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:05.389021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:05.420654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1284/2571] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:05.425324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.488984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11314c-62f9-41c2-ae78-0d3a1f82180a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.493728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.494348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.494632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.494912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.495571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.497584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.524288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1288/2572] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:05 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:05.528539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.539743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc139d-dd56-4f86-9681-5a95ab2eeb89 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.595020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.595662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.595910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.596184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.596852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.598841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.619345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1285/2573] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:05 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:48:05.623420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.634335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0acbed-c046-4b4b-ab77-2806e61b89aa admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.639123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.639726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.640012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.640293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.641074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.642703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.654700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1289/2574] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:05.658726 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.669184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72f0ed-ea83-4be7-ade0-de3e724db535 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.673864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.674522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.674808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.675094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] PATH_INFO: `/v3/projects/bddb56e8aaf54367986b5ac02e178b63/users/0574256689f645e2879eeab606b701bb/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.690127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddb56e8aaf54367986b5ac02e178b63, user_id=0574256689f645e2879eeab606b701bb, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.692996 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.706039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1286/2575] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:05 2022] PUT /identity/v3/projects/bddb56e8aaf54367986b5ac02e178b63/users/0574256689f645e2879eeab606b701bb/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:05.710078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.720097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5163f-65ba-4058-ad33-7d109ac00c45 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.749350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:05.777459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.778072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.778355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.778635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.779333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.780922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb088648-0088-499e-bf80-31657e1d937c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.791471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1290/2576] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:05.795318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.806321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d62b-62ba-4432-9d8b-1dc4f25893c3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:05.811168 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:05.811836 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.812119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.812398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] PATH_INFO: `/v3/projects/bddb56e8aaf54367986b5ac02e178b63/users/0574256689f645e2879eeab606b701bb/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.814423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddb56e8aaf54367986b5ac02e178b63, user_id=0574256689f645e2879eeab606b701bb, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:05.817101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:05.827697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1287/2577] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:05 2022] PUT /identity/v3/projects/bddb56e8aaf54367986b5ac02e178b63/users/0574256689f645e2879eeab606b701bb/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:05.840351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:05.840646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:05.840926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:05.869186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] MFA Rules not processed for user `0574256689f645e2879eeab606b701bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:05.870390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:05.937210 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1291/2578] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:05.948483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:05.950249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:06.009295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d8c9-de6f-4d2a-8ef5-d76a647e535f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:06.012882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:06.013537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:06.013823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:06.014101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:06.056823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:06.072252 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:06.073713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:06.083027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1288/2579] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:07.583103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:07.593407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d04f6-85ac-47ba-aefa-8c4a50e9bd03 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:07.597030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:07.597643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:07.597927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:07.598204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:07.653586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:07.655023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:07.664378 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1292/2580] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:08.200778 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:08.210950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8fc347-2371-4347-a866-6f0c0d3b7bc0 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:08.214710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:08.215326 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:08.215612 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:08.215894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:08.225477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:08.226937 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8ce7f-2f72-48bd-bd3a-df701e4aa370 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:08.236320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1289/2581] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:12.107849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:12.159617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:12.172551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f718dba-0d9a-4f2c-8ac8-55f482090571 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:12.177134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:12.177768 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:12.178074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:12.178349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] PATH_INFO: `/v3/users/445fc365025c4406a65d8108c2e5e655` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:12.179492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=445fc365025c4406a65d8108c2e5e655)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:12.181472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:12.226154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445fc365025c4406a65d8108c2e5e655'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.237410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445fc365025c4406a65d8108c2e5e655'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.238053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445fc365025c4406a65d8108c2e5e655'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.245456 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445fc365025c4406a65d8108c2e5e655'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.253108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445fc365025c4406a65d8108c2e5e655'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.259418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1293/2582] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:12 2022] DELETE /identity/v3/users/445fc365025c4406a65d8108c2e5e655 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:12.263926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:12.264419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:12.264536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:12.284482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:12.285669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:12.342575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1290/2583] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:12.352609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:12.354086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:12.412559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616d42b-48d0-4014-aa7c-6e78dc762ce5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:12.416358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:12.416989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:12.417277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:12.417550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:12.426529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:12.427950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:12.437207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1294/2584] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:12.696523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:12.698236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:12.712993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a90b7f-3187-4b9c-9013-e2732b098005 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:12.717784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:12.718516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:12.718803 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:12.719079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] PATH_INFO: `/v3/projects/25af0bc01ae34b6a9ba63b517385b397` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:12.721154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25af0bc01ae34b6a9ba63b517385b397)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:12.723081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:12.770154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25af0bc01ae34b6a9ba63b517385b397', 'request_id': 'req-e99c2a06-1750-4300-bee6-38d03b72c838'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.776155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25af0bc01ae34b6a9ba63b517385b397', 'request_id': 'req-e99c2a06-1750-4300-bee6-38d03b72c838'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.782452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25af0bc01ae34b6a9ba63b517385b397', 'request_id': 'req-e99c2a06-1750-4300-bee6-38d03b72c838'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.794428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] The token cache is being invalidate because project 25af0bc01ae34b6a9ba63b517385b397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:12.794722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.795336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.796819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1291/2585] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:12 2022] DELETE /identity/v3/projects/25af0bc01ae34b6a9ba63b517385b397 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:48:12.801019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:12.811322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e115c6-c110-434b-9b2a-5711d131eaf4 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:12.866755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:12.867322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:12.868141 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:12.868416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] PATH_INFO: `/v3/users/6d80f2086b174a3294250e2bb308c364` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:12.869601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d80f2086b174a3294250e2bb308c364)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:12.871673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:12.885285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:12.918187 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d80f2086b174a3294250e2bb308c364'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.929760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d80f2086b174a3294250e2bb308c364'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.930955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d80f2086b174a3294250e2bb308c364'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.937742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d80f2086b174a3294250e2bb308c364'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.944860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d80f2086b174a3294250e2bb308c364'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:12.953285 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1295/2586] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:12 2022] DELETE /identity/v3/users/6d80f2086b174a3294250e2bb308c364 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:13.224684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:13.226137 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:13.239745 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2a06-1750-4300-bee6-38d03b72c838 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:13.292955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:13.303624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:13.304276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:13.304731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:13.305941 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] PATH_INFO: `/v3/projects/2f67bafdfcbb4b999cf95fc9b09a3f3b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:13.307114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f67bafdfcbb4b999cf95fc9b09a3f3b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:13.309065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:13.363366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f67bafdfcbb4b999cf95fc9b09a3f3b', 'request_id': 'req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.369734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f67bafdfcbb4b999cf95fc9b09a3f3b', 'request_id': 'req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.375918 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f67bafdfcbb4b999cf95fc9b09a3f3b', 'request_id': 'req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.388768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] The token cache is being invalidate because project 2f67bafdfcbb4b999cf95fc9b09a3f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:13.389087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.389712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.391201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1292/2587] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:13 2022] DELETE /identity/v3/projects/2f67bafdfcbb4b999cf95fc9b09a3f3b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:13.395164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:13.396629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:13.409066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453324-9e40-4fde-b509-9459be52db7c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:13.466613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:13.467254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:13.467535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:13.467809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] PATH_INFO: `/v3/users/0574256689f645e2879eeab606b701bb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:13.469009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0574256689f645e2879eeab606b701bb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:13.470933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:13.516860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0574256689f645e2879eeab606b701bb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.528772 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0574256689f645e2879eeab606b701bb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.529467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0574256689f645e2879eeab606b701bb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.536151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0574256689f645e2879eeab606b701bb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.543586 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d0791d84-33ec-417c-b0b4-4b5c62a86e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0574256689f645e2879eeab606b701bb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.549432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1296/2588] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:48:13 2022] DELETE /identity/v3/users/0574256689f645e2879eeab606b701bb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:13.821869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:13.823486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:13.840641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7457b-90d7-4c9c-bfae-e6a3aaec7955 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:13.897946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:13.898546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:13.898834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:13.899111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] PATH_INFO: `/v3/projects/bddb56e8aaf54367986b5ac02e178b63` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:13.900284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddb56e8aaf54367986b5ac02e178b63)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:13.902249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:13.948780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddb56e8aaf54367986b5ac02e178b63', 'request_id': 'req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.954862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddb56e8aaf54367986b5ac02e178b63', 'request_id': 'req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.960947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddb56e8aaf54367986b5ac02e178b63', 'request_id': 'req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.973377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] The token cache is being invalidate because project bddb56e8aaf54367986b5ac02e178b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:48:13.973664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.974372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a84f83b9-9a4d-4f74-8b79-fb92c0df05b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:48:13.975977 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1293/2589] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:48:13 2022] DELETE /identity/v3/projects/bddb56e8aaf54367986b5ac02e178b63 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:13.981459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:13.981752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:13.982030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.007153 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:14.008584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:14.014083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:14.072448 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1297/2590] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:14.081868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.082225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.082505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.103468 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:14.104667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:14.112493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1294/2591] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:14.116987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.126890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed150840-77cc-46e5-b3d2-cd27d93e0112 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.131455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.132060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.132344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.132624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.133361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:48:14.133670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.135278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.146574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1298/2592] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:48:14.151506 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.160464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2b5e7-9fe2-4a3d-97ba-d3f06b19c04e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.165737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.166325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.166600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.166870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.167530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.169458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.173813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:14.198630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1295/2593] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:48:14 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:48:14.203435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.214315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21212a7-31f1-43d4-bac6-3b6e57cf3f01 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.280030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.280643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.280922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.281231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.281890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.283807 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.304524 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1299/2594] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:48:14 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:48:14.308887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.319633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600957c-a771-41e7-a517-7f23e3cd18e4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.324563 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.325205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.325490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.325764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.326453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.328307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.338968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1296/2595] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:14.342817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.353256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50b7f3-a20e-4b54-b353-e6fcac1162e4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.357815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.358408 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.358687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.358958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.359640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.361261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db332669-595b-43a7-84c9-634ec79df867 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.371025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1300/2596] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:48:14.375041 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.385151 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb017a8-0003-4042-bc49-4ee90884bb48 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.389691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.390346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.390630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.390910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] PATH_INFO: `/v3/projects/8d82269eab684311b711fc582aca274c/users/fa6d303e3fbc42b3b6ed98e373c8675b/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.405995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d82269eab684311b711fc582aca274c, user_id=fa6d303e3fbc42b3b6ed98e373c8675b, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.408578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.421904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1297/2597] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:48:14 2022] PUT /identity/v3/projects/8d82269eab684311b711fc582aca274c/users/fa6d303e3fbc42b3b6ed98e373c8675b/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:48:14.426759 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d18-1a71-4264-9a52-e7732130444c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.427245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d18-1a71-4264-9a52-e7732130444c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.427350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38d18-1a71-4264-9a52-e7732130444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.447207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d8e38d18-1a71-4264-9a52-e7732130444c None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:14.448478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d8e38d18-1a71-4264-9a52-e7732130444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:14.506078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1301/2598] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:14.516435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:14.518163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:14.574140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903be1-d174-4884-927d-a8b0592b081f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:14.578232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:14.578898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:14.579219 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:14.579547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:14.589637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:14.591101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:14.600541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1298/2599] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:48:15.327804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057ff5-fab2-412a-a97f-108b3acc7066 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:15.328031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057ff5-fab2-412a-a97f-108b3acc7066 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:15.328308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6057ff5-fab2-412a-a97f-108b3acc7066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:15.363412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a6057ff5-fab2-412a-a97f-108b3acc7066 None None] MFA Rules not processed for user `fa6d303e3fbc42b3b6ed98e373c8675b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:15.364743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a6057ff5-fab2-412a-a97f-108b3acc7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:15.370308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:15.440048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1302/2600] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:15.455568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:15.457101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:15.523476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed621376-578d-4975-9457-cb2914686c46 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:15.527741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:15.528372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:15.528654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:15.528929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:15.541924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:15.543534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9856a-4fc5-4d79-85ff-3e8c06ef8b4e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:15.553749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1299/2601] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:48:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:48:16.919297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032200-a876-4544-8544-d97298815f23 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:16.919830 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032200-a876-4544-8544-d97298815f23 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:16.919957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032200-a876-4544-8544-d97298815f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:16.938234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-33032200-a876-4544-8544-d97298815f23 None None] MFA Rules not processed for user `fa6d303e3fbc42b3b6ed98e373c8675b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:16.939425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-33032200-a876-4544-8544-d97298815f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:16.946748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1303/2602] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:25.548795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:26.932985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=27 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:33.301465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:33.301970 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:33.302091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:33.322378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] MFA Rules not processed for user `fa6d303e3fbc42b3b6ed98e373c8675b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:33.323630 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:33.330746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1300/2603] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:43.315818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:48:56.650031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c96ea-34d7-4118-94d4-3413ef1d3592 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:56.650533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c96ea-34d7-4118-94d4-3413ef1d3592 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:56.650649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c96ea-34d7-4118-94d4-3413ef1d3592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:56.709267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ae8c96ea-34d7-4118-94d4-3413ef1d3592 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:48:56.710462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ae8c96ea-34d7-4118-94d4-3413ef1d3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:56.772556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1304/2604] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Jul 27 20:48:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:48:56.818355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:48:56.819773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:48:56.908753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3b508-46bd-4cc1-9cd9-fd55b155745a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:48:56.913848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:48:56.914457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:48:56.914736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:48:56.915010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:48:56.923693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:48:56.925445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:48:56.934010 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1301/2605] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:48:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:49:06.769810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=44 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:06.930453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=41 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:11.383244 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:11.383704 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:11.383818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:11.403471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] MFA Rules not processed for user `f2ee302ccbdb4fa29fefc821567048e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:11.404667 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:11.411764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1305/2606] 10.222.0.65 () {58 vars in 967 bytes} [Wed Jul 27 20:49:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:11.423596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:11.434509 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba21123-e6ed-4e81-bd10-b8cb5a48e9c1 service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:11.439487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc49a64077c94d1194086179c518209e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:11.440089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:11.440404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:11.440679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:11.449447 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:11.451033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:11.459197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1302/2607] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Jul 27 20:49:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 20:49:13.364467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:13.375994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78807c7b-3874-4fe2-b8b8-2754c4b7d384 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:13.380894 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:13.381588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.381876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.382155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] PATH_INFO: `/v3/users/fa6d303e3fbc42b3b6ed98e373c8675b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.383337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6d303e3fbc42b3b6ed98e373c8675b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:13.385816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:13.433392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6d303e3fbc42b3b6ed98e373c8675b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.444930 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6d303e3fbc42b3b6ed98e373c8675b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.445628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6d303e3fbc42b3b6ed98e373c8675b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.453298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6d303e3fbc42b3b6ed98e373c8675b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.461080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-645256f2-8e64-44d0-b117-2f57dc2afe9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6d303e3fbc42b3b6ed98e373c8675b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.467981 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1306/2608] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:49:13 2022] DELETE /identity/v3/users/fa6d303e3fbc42b3b6ed98e373c8675b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:13.640221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:13.651328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf649e-59b8-48c0-9b7c-8bc866fffe7a service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:13.709477 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:13.710112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.710388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.710662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] PATH_INFO: `/v3/projects/8d82269eab684311b711fc582aca274c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.711790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d82269eab684311b711fc582aca274c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:13.714085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:13.760208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d82269eab684311b711fc582aca274c', 'request_id': 'req-176b3f42-43fa-4c92-9344-4976430e7de1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.765771 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d82269eab684311b711fc582aca274c', 'request_id': 'req-176b3f42-43fa-4c92-9344-4976430e7de1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.771772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d82269eab684311b711fc582aca274c', 'request_id': 'req-176b3f42-43fa-4c92-9344-4976430e7de1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.783642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] The token cache is being invalidate because project 8d82269eab684311b711fc582aca274c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:13.783927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.784539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-176b3f42-43fa-4c92-9344-4976430e7de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:13.786064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1303/2609] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:13 2022] DELETE /identity/v3/projects/8d82269eab684311b711fc582aca274c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:13.791290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.791690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.791985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.811766 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:13.812994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:13.874323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1307/2610] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:13.881916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.882203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.882483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.902500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:13.903701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:13.911636 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1304/2611] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:13.915801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:13.925534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40372b59-4092-47b5-bf84-7d8fe9393f78 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:13.930193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:13.930801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.931086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.931883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.932585 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:49:13.932890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:13.934523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:13.945921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1308/2612] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:49:13.950766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:13.960188 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7437695-e89c-427e-be8f-74466706faa6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:13.964897 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:13.965529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:13.965808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:13.966075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:13.966748 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:13.968640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:13.995716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1305/2613] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:49:13 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:49:13.999687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:14.010171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989480cd-6281-4f7c-8c08-7c09306b14af admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:14.058224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:14.068746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:14.069410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.069696 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.069977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.070658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:14.072812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:14.094349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1309/2614] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:49:13 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:49:14.098548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:14.109463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a23b0b-2af8-40bd-ae9b-b5edcd9b6257 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:14.114258 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:14.114862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.115141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.115412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.116098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:14.117732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:14.128399 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1306/2615] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:49:14.132261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:14.142672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72906e66-a31f-44d3-b3be-a98d43374c8d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:14.147344 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:14.147951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.148238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.148515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.149237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:14.151083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6fd31-98fc-47c0-b1c0-961e57d33aad admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:14.161503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1310/2616] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:49:14.165545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:14.175837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0c8c-b723-496f-a090-db89d6c85f2c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:14.180476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:14.181153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.182074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.182341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] PATH_INFO: `/v3/projects/2145eefa3a204900a605d4e4ecbd938e/users/6c4fd8cf92304a6dae288218cc4f665a/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.196551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2145eefa3a204900a605d4e4ecbd938e, user_id=6c4fd8cf92304a6dae288218cc4f665a, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:14.199246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:14.213433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1307/2617] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:49:14 2022] PUT /identity/v3/projects/2145eefa3a204900a605d4e4ecbd938e/users/6c4fd8cf92304a6dae288218cc4f665a/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:14.224465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.224748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.225132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.253031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] MFA Rules not processed for user `6c4fd8cf92304a6dae288218cc4f665a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:14.254221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:14.316072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1311/2618] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:49:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:49:14.327742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:14.329454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:14.344464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:14.392948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43019358-3be8-4e88-b365-472dd1378afb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:14.396522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:14.397173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:14.397453 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:14.397727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:14.406469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:14.407878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640977d-72e4-454e-98a7-47dec49dadda service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:14.416722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1308/2619] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:23.547570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:23.607934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdb4be-c0e0-46ff-8a6a-f570b86cb8c5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:23.612329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:23.612955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:23.613257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:23.613530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] PATH_INFO: `/v3/users/6c4fd8cf92304a6dae288218cc4f665a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:23.614666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4fd8cf92304a6dae288218cc4f665a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:23.616577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:23.663356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4fd8cf92304a6dae288218cc4f665a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:23.674856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:23.676025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4fd8cf92304a6dae288218cc4f665a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:23.676867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4fd8cf92304a6dae288218cc4f665a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:23.684914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4fd8cf92304a6dae288218cc4f665a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:23.692245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4fd8cf92304a6dae288218cc4f665a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:23.698543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1312/2620] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:49:23 2022] DELETE /identity/v3/users/6c4fd8cf92304a6dae288218cc4f665a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:23.703280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:23.703775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:23.703884 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:23.722768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:23.723956 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:23.782247 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1309/2621] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:49:23.792248 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:23.794314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:23.855125 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425b0f-aa6b-4bee-b3e5-4fd508d82e92 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:23.858931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:23.859393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:23.859672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:23.859951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:23.869094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:23.870520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382df76-0bf7-42db-a573-0ef797b419d0 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:23.879809 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1313/2622] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:24.148339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:24.150675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:24.164926 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a47c92-8abb-488f-8097-90324e07c8cb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:24.170580 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:24.171443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:24.172025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:24.172405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] PATH_INFO: `/v3/projects/2145eefa3a204900a605d4e4ecbd938e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:24.173764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2145eefa3a204900a605d4e4ecbd938e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:24.176107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:24.224297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2145eefa3a204900a605d4e4ecbd938e', 'request_id': 'req-839bf08d-b295-43b4-9b3e-9601e4b9c27d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:24.229812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2145eefa3a204900a605d4e4ecbd938e', 'request_id': 'req-839bf08d-b295-43b4-9b3e-9601e4b9c27d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:24.235824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2145eefa3a204900a605d4e4ecbd938e', 'request_id': 'req-839bf08d-b295-43b4-9b3e-9601e4b9c27d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:24.248120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] The token cache is being invalidate because project 2145eefa3a204900a605d4e4ecbd938e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:49:24.248406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:24.249048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:49:24.250528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1310/2623] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:49:24 2022] DELETE /identity/v3/projects/2145eefa3a204900a605d4e4ecbd938e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:49:33.875600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:34.245218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:36.407452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:36.408013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:36.408013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:36.427253 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] MFA Rules not processed for user `30051341684845938161bb5ec0680d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:49:36.428451 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:36.482870 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1314/2624] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:49:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:49:36.784420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:49:36.785943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:36.843319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf08d-b295-43b4-9b3e-9601e4b9c27d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:49:36.846868 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:49:36.847482 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:49:36.847762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:49:36.848034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:49:36.849807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:49:36.859655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:49:36.861103 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:49:36.870608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1311/2625] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:49:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:49:46.475223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=25 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:49:46.873275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=26 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:50:03.852131 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:50:03.854164 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:50:03.931510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a156f45-1dd8-4776-9ede-a93142955ec3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:50:03.935169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:50:03.935817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:50:03.936100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:50:03.936376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:50:03.938157 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:50:04.022572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:50:04.024034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:50:04.042365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1315/2626] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:50:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:50:14.039256 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=70 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:04.080548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:04.082657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:04.204364 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1846-9472-4854-a014-e96d54193205 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:04.210146 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:04.210774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:04.211053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:04.211335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:04.213851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:04.224347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:04.226001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:04.237496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1312/2627] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:05.370683 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:05.372201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:05.393654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af7917-37e9-4dff-b1c1-14fb1776d990 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:05.397311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:05.397918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:05.398202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:05.398478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:05.400868 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:05.413982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:05.415906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:05.433014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1316/2628] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:05.652743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:05.671504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd52dd-f248-4379-b247-d72fe6942874 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:05.675104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:05.675732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:05.676012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:05.676285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:05.678058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:05.691340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:05.692742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:05.704850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1313/2629] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:06.916433 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:06.918080 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:06.933123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77587b88-7ff2-4c9a-b320-5624d5581cc1 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:06.938762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:06.939374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:06.939655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:06.939931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:06.941698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:06.955424 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:06.957072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:06.968037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1317/2630] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:10.359898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:10.369120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdaea1-d5ac-4d61-8efc-b9c6a941a4b1 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:10.372730 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:10.373363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:10.373641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:10.373911 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:10.375626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:10.437122 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:10.438560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:10.448284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1314/2631] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:13.327326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:13.328726 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:13.386464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:13.388584 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:13.423165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecdb10-3ee3-44de-9b9f-b5df9f58a544 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:13.429321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:13.430294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:13.430588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:13.430962 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] PATH_INFO: `/v3/users/a48eaef4d70b4116bd307374d819bca6` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:13.432422 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48eaef4d70b4116bd307374d819bca6)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:13.435191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:13.442032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b5b2f-0db2-421b-9d7e-00518c90e026 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:13.456201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:13.456355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:13.457298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:13.457540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:13.457983 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:13.460114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:13.477844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:13.480614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:13.491746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1318/2632] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:13.508904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48eaef4d70b4116bd307374d819bca6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.508904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48eaef4d70b4116bd307374d819bca6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.527872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48eaef4d70b4116bd307374d819bca6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.542541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48eaef4d70b4116bd307374d819bca6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.543284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9a2b9c6c-adfb-4195-8243-685ca205f22f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48eaef4d70b4116bd307374d819bca6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.550802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1315/2633] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:13 2022] DELETE /identity/v3/users/a48eaef4d70b4116bd307374d819bca6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:13.741325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:13.754484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a908-27aa-4150-b04a-d03b8b8ce4d9 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:13.821155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:13.821798 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:13.822675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:13.823098 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] PATH_INFO: `/v3/projects/1626b826d47a47429a947fa36ace64e4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:13.824529 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] RBAC: Authorizing `identity:delete_project(project_id=1626b826d47a47429a947fa36ace64e4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:13.826898 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:13.875713 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1626b826d47a47429a947fa36ace64e4', 'request_id': 'req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.880466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:13.887698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1626b826d47a47429a947fa36ace64e4', 'request_id': 'req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.896668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1626b826d47a47429a947fa36ace64e4', 'request_id': 'req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.911254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] The token cache is being invalidate because project 1626b826d47a47429a947fa36ace64e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:13.911591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.912225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbdc0984-bfc1-439a-b16e-b1b4c34efdee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:13.914286 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1319/2634] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:13 2022] DELETE /identity/v3/projects/1626b826d47a47429a947fa36ace64e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:13.920383 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:13.920675 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:13.921064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:13.943107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:13.944622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:14.003929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1316/2635] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:14.013172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.013465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.013748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.033754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:14.034964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:14.043779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1320/2636] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:14.047896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.057518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb84176-c881-47df-bedb-30f97b3707b9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.062006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.062606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.062882 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.063153 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.063838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:51:14.064139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.065852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.078551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1317/2637] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:51:14.082921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.092925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00400b-cdeb-4f41-aa47-9502b133b3ac None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.097729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.098287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.098572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.098848 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.099519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.101482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.129947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1321/2638] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:51:14.134060 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.144604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16fae-0a49-4fa8-86d0-8e655ad54d56 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.203199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.203870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.204264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.204591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.205305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.207237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.229719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1318/2639] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:51:14.233865 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.244952 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a41ac8-414c-43e6-921d-7c9147dbeeba admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.249862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.250467 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.250742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.251014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.251697 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.253325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.263069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1322/2640] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:14.266989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.277786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8cfb9-c9df-4533-9b61-b1f432c656d6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.285185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.285797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.286078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.286354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.287044 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.289215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20620289-26c3-4772-a93f-afe9541b2d48 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.295236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:14.301400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1319/2641] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:14.305169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.315024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae945ab1-c443-46ba-90b6-7fb4f5e21280 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.319598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.320268 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.320547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.320818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] PATH_INFO: `/v3/projects/9a37e1e5f3574166b4d61529c1849375/users/e07a29ce734e46868092f22c92e63122/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.335724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a37e1e5f3574166b4d61529c1849375, user_id=e07a29ce734e46868092f22c92e63122, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.339044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.351862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1323/2642] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:14 2022] PUT /identity/v3/projects/9a37e1e5f3574166b4d61529c1849375/users/e07a29ce734e46868092f22c92e63122/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:14.362259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.362552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.362855 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.391066 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] MFA Rules not processed for user `e07a29ce734e46868092f22c92e63122`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:14.392289 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:14.458840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1320/2643] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:14.461927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.525074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17abf1-2655-4a16-844b-52b45e4298d6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.530215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.530832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.531117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.531394 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.532051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.533993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.568254 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1324/2644] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:14.572561 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.583362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c00a-d278-4a79-9433-0f1ec23601b5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.639505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.640128 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.640415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.640687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.641371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.643275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.664191 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1321/2645] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:51:14.669089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.679400 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64a60b-d3e3-45b3-9c9a-cffe9dc2de1e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.685137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.685745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.686030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.686309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.687009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.688603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.701039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1325/2646] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:14.705246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.716272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb9cbf-11e3-4bdc-9332-c383aea93755 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.720955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.721650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.721933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.722204 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] PATH_INFO: `/v3/projects/eb1719dcc2ff4650a16ab547c9642479/users/f1a667d8a6f34dba9cff8fa08a1e4e2f/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.738367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1719dcc2ff4650a16ab547c9642479, user_id=f1a667d8a6f34dba9cff8fa08a1e4e2f, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.741079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.754920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1322/2647] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:14 2022] PUT /identity/v3/projects/eb1719dcc2ff4650a16ab547c9642479/users/f1a667d8a6f34dba9cff8fa08a1e4e2f/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:14.759061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.769551 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00c766-3cf8-4898-a370-4d58c555dbfd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.819369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:14.828693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.830208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.830495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.830776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.831478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.833124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b61b-3ecb-4478-a2df-42341a03f8f7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.845087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1326/2648] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:51:14.849174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:14.860634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882a9a9-3f40-4950-be2b-b11c6c0446da admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:14.865576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:14.866365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.866649 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.866929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] PATH_INFO: `/v3/projects/eb1719dcc2ff4650a16ab547c9642479/users/f1a667d8a6f34dba9cff8fa08a1e4e2f/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.868938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1719dcc2ff4650a16ab547c9642479, user_id=f1a667d8a6f34dba9cff8fa08a1e4e2f, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:14.871722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:14.882499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1323/2649] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:14 2022] PUT /identity/v3/projects/eb1719dcc2ff4650a16ab547c9642479/users/f1a667d8a6f34dba9cff8fa08a1e4e2f/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:14.893347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dab13-3098-4749-8942-78b09cb35035 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:14.893564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dab13-3098-4749-8942-78b09cb35035 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:14.893844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dab13-3098-4749-8942-78b09cb35035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:14.924282 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-350dab13-3098-4749-8942-78b09cb35035 None None] MFA Rules not processed for user `f1a667d8a6f34dba9cff8fa08a1e4e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:14.925554 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-350dab13-3098-4749-8942-78b09cb35035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:14.994996 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1327/2650] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:14 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:15.008355 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:15.010425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:15.059148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:15.090215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116920b-51bb-4f21-95ed-2d1a8b561484 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:15.094033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:15.094652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:15.094936 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:15.095211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:15.105480 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:15.106909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917acc0-bd08-4bba-ad67-9a6d83461b61 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:15.116394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1324/2651] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:15 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:15.196272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350dab13-3098-4749-8942-78b09cb35035 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:15.209085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350dab13-3098-4749-8942-78b09cb35035 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:15.213439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:15.214137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:15.214485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:15.214832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:15.274566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:15.276810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:15.286092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1328/2652] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:21.764365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:21.764837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:21.764837 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:21.783570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] MFA Rules not processed for user `30051341684845938161bb5ec0680d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:21.784780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:21.843705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1325/2653] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:23.506374 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:23.507934 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:23.623845 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad1b9-ea91-4941-bade-b1faa31bb266 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:23.629680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:23.630325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:23.630606 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:23.630885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:23.632837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:23.646822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:23.658060 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:23.679147 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1329/2654] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:28.501022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:28.502542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:28.570520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0384-5219-4014-91db-2994405ac105 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:28.574757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:28.575441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:28.575721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:28.575993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:28.578067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:28.588332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:28.589960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f333ad-5149-46dc-bcf2-f1a118ea3f1f service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:28.601784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1326/2655] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:30.291680 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:30.297116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:30.358620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805bfe3-42f6-4cc2-9f37-9da81ed069b3 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:30.363308 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:30.363979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:30.364291 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:30.364563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] PATH_INFO: `/v3/users/e07a29ce734e46868092f22c92e63122` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:30.365816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07a29ce734e46868092f22c92e63122)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:30.367735 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:30.417982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07a29ce734e46868092f22c92e63122'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.429913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07a29ce734e46868092f22c92e63122'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.430746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07a29ce734e46868092f22c92e63122'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.439781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07a29ce734e46868092f22c92e63122'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.449724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07a29ce734e46868092f22c92e63122'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.456018 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1330/2656] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:30 2022] DELETE /identity/v3/users/e07a29ce734e46868092f22c92e63122 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:30.460473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:30.460986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:30.461076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:30.479465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:30.480652 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:30.537591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1327/2657] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:51:30.547339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:30.548786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:30.605107 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08741e95-bd6e-4f09-b059-71f16ad859c4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:30.608684 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:30.609315 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:30.609591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:30.609860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:30.618669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:30.620702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:30.629914 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1331/2658] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:30.896834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:30.898338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:30.911378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83e6a4-ff1a-40ef-b50d-b8c72092d1af None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:30.916922 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:30.918114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:30.918384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:30.918871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] PATH_INFO: `/v3/projects/9a37e1e5f3574166b4d61529c1849375` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:30.920279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a37e1e5f3574166b4d61529c1849375)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:30.922226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:30.941346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:30.971320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a37e1e5f3574166b4d61529c1849375', 'request_id': 'req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.976689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a37e1e5f3574166b4d61529c1849375', 'request_id': 'req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.983330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a37e1e5f3574166b4d61529c1849375', 'request_id': 'req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.995099 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] The token cache is being invalidate because project 9a37e1e5f3574166b4d61529c1849375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:30.995391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.996555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:30.998164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1328/2659] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:30 2022] DELETE /identity/v3/projects/9a37e1e5f3574166b4d61529c1849375 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:31.002181 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.011823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56e046-a274-4723-9ea1-95425de28830 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.034997 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:31.070045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.070635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.070917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.071196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] PATH_INFO: `/v3/users/f1a667d8a6f34dba9cff8fa08a1e4e2f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.072384 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a667d8a6f34dba9cff8fa08a1e4e2f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.074477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.124587 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a667d8a6f34dba9cff8fa08a1e4e2f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.136121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a667d8a6f34dba9cff8fa08a1e4e2f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.136955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a667d8a6f34dba9cff8fa08a1e4e2f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.143717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a667d8a6f34dba9cff8fa08a1e4e2f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.151654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-81b8535a-7b71-48fa-98ce-af8b5c0545f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a667d8a6f34dba9cff8fa08a1e4e2f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.158049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1332/2660] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:31 2022] DELETE /identity/v3/users/f1a667d8a6f34dba9cff8fa08a1e4e2f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:31.425587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.427118 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:31.440198 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b83e3-4864-4e9f-bb71-bf1c14b7dbd9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.501927 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.502610 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.502847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.503126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] PATH_INFO: `/v3/projects/eb1719dcc2ff4650a16ab547c9642479` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.504388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1719dcc2ff4650a16ab547c9642479)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.506351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.558163 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1719dcc2ff4650a16ab547c9642479', 'request_id': 'req-8f5393ac-7434-467b-9577-b25ea7ff08a8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.563864 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1719dcc2ff4650a16ab547c9642479', 'request_id': 'req-8f5393ac-7434-467b-9577-b25ea7ff08a8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.570070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1719dcc2ff4650a16ab547c9642479', 'request_id': 'req-8f5393ac-7434-467b-9577-b25ea7ff08a8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.584079 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] The token cache is being invalidate because project eb1719dcc2ff4650a16ab547c9642479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:31.584367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.585020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f5393ac-7434-467b-9577-b25ea7ff08a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:31.586554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1329/2661] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:31 2022] DELETE /identity/v3/projects/eb1719dcc2ff4650a16ab547c9642479 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:31.592231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.592706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.592885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.613967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:31.615171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:31.674354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1333/2662] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:31.681950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.682233 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.682503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.703025 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:31.704653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:31.712743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1330/2663] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:31.717539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.726958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6525df2-d83a-4459-bf2d-5780c833df1d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.731557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.732148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.732426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.732698 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.733418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:51:31.733724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.735312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.746479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1334/2664] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:51:31.750556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.759585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e295056-ef50-45ee-8a39-cfd6b65a274d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.764769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.765395 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.765674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.765944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.766600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.768475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.794825 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1331/2665] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:31 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:31.798822 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.809319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e905-4ca5-4085-94f9-1e86404c05d1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.839172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:31.867100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.867641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.867927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.868208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.868890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.871396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.893276 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1335/2666] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:31 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:31.897633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.908343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f80d-10d9-4bcc-8e91-e812f283361f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.912915 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.913550 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.913831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.914104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.914790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.916382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.917635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:31.927396 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1332/2667] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:31.931405 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.942204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5cf38-593f-4744-a8d2-c7d97c7e53b9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.947034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.947688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.948008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.948336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:31.949811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:31.952004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf7b36-ba8a-4363-ab1d-6c36336ade5c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:31.963097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1336/2668] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:31.967800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:31.977904 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd24d64-4cf2-473e-84b8-52d7886e52e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:31.983120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:31.983793 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:31.984075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:31.984352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] PATH_INFO: `/v3/projects/452acae814664f90af47a52d6245d0a1/users/bdfa505ecf0740ad961b02ffebb61e3b/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.001225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=452acae814664f90af47a52d6245d0a1, user_id=bdfa505ecf0740ad961b02ffebb61e3b, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.004022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.017981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1333/2669] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:31 2022] PUT /identity/v3/projects/452acae814664f90af47a52d6245d0a1/users/bdfa505ecf0740ad961b02ffebb61e3b/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:51:32.028629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.029002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.029310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.057290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] MFA Rules not processed for user `bdfa505ecf0740ad961b02ffebb61e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:32.059167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:32.135028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1337/2670] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:32.140031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.203921 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dbaf25-368c-4fd8-9fe2-5fdec24ea2fe admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.213616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.214269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.214763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.214983 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.215681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.219973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.249097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1334/2671] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:32 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:32.253274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.264763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f5f2c-8e1e-44a9-8b18-80754736aa3a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.322184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.323050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.323336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.323613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.324567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.328431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.350965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1338/2672] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:32 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:32.355520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.366227 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694e0c2-89c3-4e20-99b3-7e7fc75d160d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.371438 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.372327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.372646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.372946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.373667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.375317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.385859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1335/2673] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:32.390128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.400367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637717e-af9b-408d-9472-2e892d0680cb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.405310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.405984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.406271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.406552 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] PATH_INFO: `/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee/users/05d8bf64ca064f3c83ed1c425eaa0da6/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.422012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfb8c8a99ff42e38a97d55e780177ee, user_id=05d8bf64ca064f3c83ed1c425eaa0da6, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.424729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.438541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1339/2674] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:32 2022] PUT /identity/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee/users/05d8bf64ca064f3c83ed1c425eaa0da6/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:32.442659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.453405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390cae3-39a9-422a-9fdb-5ee79835af27 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.515415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.516084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.516293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.516565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.517308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.518891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb36d4-016f-41e3-ac90-d6c723c57ce3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.529446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1336/2675] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:32.533628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.544763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33be3f9-d4a2-4da6-a8bc-9db335b3e878 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.549480 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.550134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.550416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.550693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] PATH_INFO: `/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee/users/05d8bf64ca064f3c83ed1c425eaa0da6/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.552728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfb8c8a99ff42e38a97d55e780177ee, user_id=05d8bf64ca064f3c83ed1c425eaa0da6, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.555402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.557829 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:32.572823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1340/2676] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:32 2022] PUT /identity/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee/users/05d8bf64ca064f3c83ed1c425eaa0da6/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:32.586522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.586806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.587083 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.615967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] MFA Rules not processed for user `05d8bf64ca064f3c83ed1c425eaa0da6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:32.617281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:32.647542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:32.687869 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1337/2677] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:32.698993 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.700800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:32.760047 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d79b-7cfb-44bf-8106-eb42a329b188 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.763999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.764624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.764908 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.765211 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.774496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.775917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.786593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1341/2678] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:32.863009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:32.872687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967735f-85aa-46b8-9ea3-2d66a3cd2c9b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:32.876448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:32.877299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:32.877583 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:32.877857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:32.938235 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:32.939586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:32.948430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1338/2679] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:35.684310 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:35.686306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:35.773892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb88e-e00a-49c4-ba85-f854b129550a service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:35.780099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:35.780717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:35.781020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:35.781302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:35.783078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:35.796116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:35.797731 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:35.809335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1342/2680] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:42.944374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=43 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:44.628474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:44.630085 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:44.646230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91d0f8-c195-4372-8caa-eb2bbb055ebd service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:44.651838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:44.652454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:44.652735 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:44.653712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:44.656172 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:44.669378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:44.671081 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e72ab-e7b5-4b99-a51d-98be62c004ad service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:44.681663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1339/2681] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:45.400100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:45.461511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdfefd-643b-4231-b491-0728ebf44985 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:45.466984 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:45.467632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:45.467976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:45.468324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] PATH_INFO: `/v3/users/bdfa505ecf0740ad961b02ffebb61e3b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:45.469539 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfa505ecf0740ad961b02ffebb61e3b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:45.471616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:45.497687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:45.525622 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfa505ecf0740ad961b02ffebb61e3b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:45.544105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfa505ecf0740ad961b02ffebb61e3b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:45.545258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfa505ecf0740ad961b02ffebb61e3b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:45.553183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfa505ecf0740ad961b02ffebb61e3b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:45.561053 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfa505ecf0740ad961b02ffebb61e3b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:45.571323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1343/2682] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:45 2022] DELETE /identity/v3/users/bdfa505ecf0740ad961b02ffebb61e3b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:45.572991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d566b01-647f-4435-8367-d4742f557aef None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:45.573312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d566b01-647f-4435-8367-d4742f557aef None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:45.573593 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d566b01-647f-4435-8367-d4742f557aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:45.593577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2d566b01-647f-4435-8367-d4742f557aef None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:45.594809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2d566b01-647f-4435-8367-d4742f557aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:45.653953 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1340/2683] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:51:45.663594 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:45.665225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:45.727028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa4fba-a824-49d8-a4df-26c35ef7f4df admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:45.730785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:45.731563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:45.731853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:45.732134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:45.741307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:45.742751 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:45.752289 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1344/2684] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:46.022758 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d566b01-647f-4435-8367-d4742f557aef None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.024203 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2d566b01-647f-4435-8367-d4742f557aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:46.038317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d566b01-647f-4435-8367-d4742f557aef None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:46.044369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:46.045058 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.045412 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.045606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] PATH_INFO: `/v3/projects/452acae814664f90af47a52d6245d0a1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.046933 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=452acae814664f90af47a52d6245d0a1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:46.048845 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:46.098049 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452acae814664f90af47a52d6245d0a1', 'request_id': 'req-ac0ee901-2988-4a60-83c0-f343af00c694'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.103498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452acae814664f90af47a52d6245d0a1', 'request_id': 'req-ac0ee901-2988-4a60-83c0-f343af00c694'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.110466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452acae814664f90af47a52d6245d0a1', 'request_id': 'req-ac0ee901-2988-4a60-83c0-f343af00c694'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.124108 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] The token cache is being invalidate because project 452acae814664f90af47a52d6245d0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:46.124403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.125064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.126578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1341/2685] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:46 2022] DELETE /identity/v3/projects/452acae814664f90af47a52d6245d0a1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:46.130609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.140312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4f36-2e98-4afd-8f4f-bb8af651bf60 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:46.195746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:46.196381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.196662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.196940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] PATH_INFO: `/v3/users/05d8bf64ca064f3c83ed1c425eaa0da6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.198120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d8bf64ca064f3c83ed1c425eaa0da6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:46.200228 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:46.243584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:46.245287 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d8bf64ca064f3c83ed1c425eaa0da6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.258013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d8bf64ca064f3c83ed1c425eaa0da6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.258662 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d8bf64ca064f3c83ed1c425eaa0da6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.265695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d8bf64ca064f3c83ed1c425eaa0da6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.272843 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1dd06422-05ce-43a7-9a3e-50abe3cfd898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d8bf64ca064f3c83ed1c425eaa0da6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.279371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1345/2686] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:46 2022] DELETE /identity/v3/users/05d8bf64ca064f3c83ed1c425eaa0da6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:46.556076 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.557555 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:46.572852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ee901-2988-4a60-83c0-f343af00c694 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:46.632867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:46.633469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.633754 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.634035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] PATH_INFO: `/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.635184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfb8c8a99ff42e38a97d55e780177ee)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:46.637670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:46.648912 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:46.686402 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfb8c8a99ff42e38a97d55e780177ee', 'request_id': 'req-dc7b5019-d565-4829-a542-c96111bf37b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.692148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfb8c8a99ff42e38a97d55e780177ee', 'request_id': 'req-dc7b5019-d565-4829-a542-c96111bf37b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.698285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfb8c8a99ff42e38a97d55e780177ee', 'request_id': 'req-dc7b5019-d565-4829-a542-c96111bf37b5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.711309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] The token cache is being invalidate because project 3cfb8c8a99ff42e38a97d55e780177ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:46.711625 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.712389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dc7b5019-d565-4829-a542-c96111bf37b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:46.713974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1342/2687] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:46 2022] DELETE /identity/v3/projects/3cfb8c8a99ff42e38a97d55e780177ee => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:46.719615 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.720074 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.720180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.740043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:46.741267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:46.801764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1346/2688] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:51:46.810093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.810404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.810680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.831448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:46.832656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:46.841170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1343/2689] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:46.845330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.855029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e1ff4-de99-4d9d-ad8b-caa649f91767 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:46.860324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:46.860928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.861257 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.861536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.862231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:51:46.862536 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:46.864138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:46.875674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1347/2690] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:51:46.879671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.889111 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b352f3e-2637-4457-ba47-90b4654bf213 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:46.893770 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:46.894365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:46.894645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:46.894919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:46.895576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:46.897487 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:46.925360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1344/2691] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:46 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:46.929558 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:46.939687 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c302c-45f6-4e17-b0ca-59d9fddecdd9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:47.122567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:47.123186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.123465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.123741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.124403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:47.126330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:47.146985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1348/2692] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:46 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:51:47.151071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:47.162143 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7e27e-de2e-4fa3-bc8b-e8282e156e5e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:47.166832 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:47.167500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.167782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.168054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.169637 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:47.171279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:47.181476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1345/2693] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:47.185501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:47.195367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea8da0-cfbf-4b9a-a39d-b3fa90c3f0da admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:47.200763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:47.201692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.201979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.202259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.202950 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:47.204537 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ba537-500b-4ab1-971c-8eb0aac77f8d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:47.214560 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1349/2694] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:47.219314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:47.229297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5722-43c5-482e-89f7-22feab0d313d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:47.233883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:47.234531 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.234813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.235089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] PATH_INFO: `/v3/projects/c815bcc774f348be866862481db10816/users/f8cc7ba3c0a94d1db1f9721c780e1001/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.249273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c815bcc774f348be866862481db10816, user_id=f8cc7ba3c0a94d1db1f9721c780e1001, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:47.251895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:47.265305 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1346/2695] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:47 2022] PUT /identity/v3/projects/c815bcc774f348be866862481db10816/users/f8cc7ba3c0a94d1db1f9721c780e1001/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:51:47.276819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.277142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.277418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.304318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] MFA Rules not processed for user `f8cc7ba3c0a94d1db1f9721c780e1001`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:47.305617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:47.321980 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:47.372103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1350/2696] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:47.382615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:47.384284 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:47.439797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a0b1f-1530-4e63-8a1a-108dfef3baac admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:47.443332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:47.443932 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:47.444207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:47.444478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:47.453242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:47.454648 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b73d7b-78f4-440a-aaff-5f00edede941 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:47.463223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1347/2697] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:51:57.077021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:57.138821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a4d08-079d-4d62-a813-16f4a1f0c180 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:57.143307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:57.143937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.144218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.144496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] PATH_INFO: `/v3/users/f8cc7ba3c0a94d1db1f9721c780e1001` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.145619 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cc7ba3c0a94d1db1f9721c780e1001)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:57.147530 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:57.192442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cc7ba3c0a94d1db1f9721c780e1001'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.204547 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cc7ba3c0a94d1db1f9721c780e1001'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.205227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cc7ba3c0a94d1db1f9721c780e1001'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.211905 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cc7ba3c0a94d1db1f9721c780e1001'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.219087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cc7ba3c0a94d1db1f9721c780e1001'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.224879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1351/2698] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:51:57 2022] DELETE /identity/v3/users/f8cc7ba3c0a94d1db1f9721c780e1001 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:51:57.229348 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.229834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.229923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.249492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:57.250668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:57.283189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:57.311205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1348/2699] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:57.320854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:57.322309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:57.363703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:57.382590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c1be1-8e5d-4d05-8906-2cdef365b910 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:57.386180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:57.386818 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.387093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.387364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.396302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:57.397728 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf20f97-54e1-4f62-8e59-7a9935abc4b0 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:57.407493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1352/2700] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:51:57.682668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:57.684084 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:57.699680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25944eba-6aa7-469b-a135-2aed9180b186 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:57.705310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:57.705987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.706268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.706543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] PATH_INFO: `/v3/projects/c815bcc774f348be866862481db10816` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.707827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c815bcc774f348be866862481db10816)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:57.709672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:57.756661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c815bcc774f348be866862481db10816', 'request_id': 'req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.762212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c815bcc774f348be866862481db10816', 'request_id': 'req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.768686 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c815bcc774f348be866862481db10816', 'request_id': 'req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.781339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] The token cache is being invalidate because project c815bcc774f348be866862481db10816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:51:57.781624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.782237 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2306b39e-5dfd-4bef-9b86-2c693f70bdb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:51:57.783719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1349/2701] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:51:57 2022] DELETE /identity/v3/projects/c815bcc774f348be866862481db10816 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:51:57.788589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.788874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.789188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.811207 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:57.812438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:57.871170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1353/2702] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:57.879222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.879520 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.879800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.899807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:57.901109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:57.909949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1350/2703] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:57.914024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:57.923584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758d95-65bc-416e-a208-dadcbfd1eff1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:57.928224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:57.928827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.929221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.929493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.930179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:51:57.930484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:57.932068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:57.943203 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1354/2704] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:51:57.948938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:57.959688 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdca849-245a-4531-b70e-f677c7a714fb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:57.964352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:57.964985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:57.965273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:57.965547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:57.967045 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:57.969120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:57.997038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1351/2705] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:51:57 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:51:58.001995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.011660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5303-dfa1-4e16-8008-4ecbf41cb9b8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.065975 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.066575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.066851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.067121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.067770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.070202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.092052 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1355/2706] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:51:58 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:51:58.096106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.106259 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d951e1-aeba-4320-88c6-0e526e17f38e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.111448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.112046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.112323 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.112592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.113320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.114892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.124761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1352/2707] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:58.128509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.138359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b2d35-d51e-418e-bf01-2967d4a4933f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.143191 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.143785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.144063 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.144336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.145064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.146621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe49735-94e0-4e2e-803b-1ec788df9631 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.157088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1356/2708] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:51:58.160950 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.171623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdccefe-162e-4eed-9b83-795068c567f7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.176169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.176879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.177189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.177529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] PATH_INFO: `/v3/projects/a6b9f7b4c94b4fbfb35e995fdd63f304/users/0494406bcb0e4bc4a0b035a77fe6c918/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.192335 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b9f7b4c94b4fbfb35e995fdd63f304, user_id=0494406bcb0e4bc4a0b035a77fe6c918, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.194991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.207304 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1353/2709] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:51:58 2022] PUT /identity/v3/projects/a6b9f7b4c94b4fbfb35e995fdd63f304/users/0494406bcb0e4bc4a0b035a77fe6c918/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:51:58.217871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.218155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.218427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.246295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] MFA Rules not processed for user `0494406bcb0e4bc4a0b035a77fe6c918`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:51:58.247517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:58.279204 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:51:58.310372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1357/2710] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:51:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:51:58.320902 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:51:58.322635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:51:58.378374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3705eb-b223-4f98-bdf9-0138bc88b34d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:51:58.381909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:51:58.382516 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:51:58.382797 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:51:58.383073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:51:58.391270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:51:58.392707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a2640-d6d2-4c9c-b960-a86426d02f50 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:51:58.401340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1354/2711] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:51:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:01.391692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:01.453086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136dbdbb-1ede-4927-bdf9-f5f45bb7bab3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:01.457618 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:01.458255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:01.458538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:01.458815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] PATH_INFO: `/v3/users/0494406bcb0e4bc4a0b035a77fe6c918` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:01.460012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0494406bcb0e4bc4a0b035a77fe6c918)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:01.461962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:01.506473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0494406bcb0e4bc4a0b035a77fe6c918'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:01.518002 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0494406bcb0e4bc4a0b035a77fe6c918'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:01.518659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0494406bcb0e4bc4a0b035a77fe6c918'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:01.525390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0494406bcb0e4bc4a0b035a77fe6c918'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:01.533600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0494406bcb0e4bc4a0b035a77fe6c918'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:01.539403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1358/2712] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:01 2022] DELETE /identity/v3/users/0494406bcb0e4bc4a0b035a77fe6c918 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:01.543917 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:01.544360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:01.544475 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:01.552230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:01.566160 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:01.567366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:01.626347 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1355/2713] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:01.636049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:01.637517 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:01.693354 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619cad-d0d4-449a-a377-73dc38c335e7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:01.696991 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:01.697617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:01.697900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:01.698176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:01.707860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:01.709320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce9556-16cc-4a80-b28a-1ac48bca2edb service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:01.713246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:01.725139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1359/2714] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:01.989752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:01.991328 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:02.004249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1721133-144f-4deb-add2-37d123188fe2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.009087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.009769 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.010046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.010319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] PATH_INFO: `/v3/projects/a6b9f7b4c94b4fbfb35e995fdd63f304` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.011532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b9f7b4c94b4fbfb35e995fdd63f304)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.013490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.058945 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b9f7b4c94b4fbfb35e995fdd63f304', 'request_id': 'req-1b578379-65b9-463f-ae24-7901f92bc77b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:02.065069 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b9f7b4c94b4fbfb35e995fdd63f304', 'request_id': 'req-1b578379-65b9-463f-ae24-7901f92bc77b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:02.071154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b9f7b4c94b4fbfb35e995fdd63f304', 'request_id': 'req-1b578379-65b9-463f-ae24-7901f92bc77b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:02.084242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] The token cache is being invalidate because project a6b9f7b4c94b4fbfb35e995fdd63f304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:02.084533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:02.085185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b578379-65b9-463f-ae24-7901f92bc77b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:02.086681 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1356/2715] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:01 2022] DELETE /identity/v3/projects/a6b9f7b4c94b4fbfb35e995fdd63f304 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:02.093977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.094261 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.094538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.113265 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:02.114440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:02.173982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1360/2716] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:02.181452 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.181743 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.182026 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.202208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:02.203414 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:02.211294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1357/2717] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:02.215331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.224702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdf38d-40f6-48d1-a766-e5e8ca793284 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.229205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.229797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.230076 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.230350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.231025 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:02.231324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.233655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.245096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1361/2718] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:52:02.249641 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.258698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa331-b04a-46be-a41c-2c5190390fa9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.263336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.263997 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.264490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.264782 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.265497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.267386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.294155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1358/2719] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:02 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:02.298879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.308771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617651-e4ee-4674-9097-3235c5c7ad0a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.363922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.364520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.364802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.365113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.365765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.367681 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.388221 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1362/2720] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:02 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:02.392306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.402582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dd567-8fe9-4766-a6ef-d04c643f4c4b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.408607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.410377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.410674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.410958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.411659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.413331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.426249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1359/2721] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:02.430142 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.440819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dd3bf-ce75-47cf-a726-2a172eb3a8bb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.445639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.446241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.446523 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.446792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.447474 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.449105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f639c-d995-48a7-ac25-a931cc4749cf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.459700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1363/2722] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:02.463646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.473694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7c018-5dbf-406a-8638-2008603e37ab admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.478366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.479021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.479308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.479634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] PATH_INFO: `/v3/projects/901b5f7243354f529ad1ed560b3703a3/users/2c4a2d31623f40c783d88eba3f90f094/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.494990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901b5f7243354f529ad1ed560b3703a3, user_id=2c4a2d31623f40c783d88eba3f90f094, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.497685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.511344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1360/2723] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:02 2022] PUT /identity/v3/projects/901b5f7243354f529ad1ed560b3703a3/users/2c4a2d31623f40c783d88eba3f90f094/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:02.521668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.521939 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.522214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.549593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] MFA Rules not processed for user `2c4a2d31623f40c783d88eba3f90f094`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:02.550816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:02.603351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:02.614931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1364/2724] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:02.626807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:02.628530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:02.655075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:02.687792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db9a7-8cf5-4d72-9cdd-cf344e295727 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:02.691297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:02.691909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:02.692186 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:02.692455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:02.700486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:02.702208 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b8690-807c-4be6-bcda-97426c0df057 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:02.710779 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1361/2725] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:12.612008 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:12.706907 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=16 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:48.655138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.716584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e0573-1220-486f-9377-52b58aa388c6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.721180 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.722459 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.723026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.723026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] PATH_INFO: `/v3/users/2c4a2d31623f40c783d88eba3f90f094` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.724205 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4a2d31623f40c783d88eba3f90f094)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.726168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.773266 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4a2d31623f40c783d88eba3f90f094'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.785355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4a2d31623f40c783d88eba3f90f094'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.786017 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4a2d31623f40c783d88eba3f90f094'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.792694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4a2d31623f40c783d88eba3f90f094'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.799591 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4a2d31623f40c783d88eba3f90f094'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:48.805572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1365/2726] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:52:48 2022] DELETE /identity/v3/users/2c4a2d31623f40c783d88eba3f90f094 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:48.810230 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.810721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.810842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.829787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:48.831491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.888292 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1362/2727] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:48.898493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:48.899876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:48.955844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82116e-7c6b-4e16-bf24-b53c54a8db30 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:48.960048 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:48.960651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:48.960925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:48.961229 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:48.970449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:48.972023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fd0f0-948c-4631-832c-6a7937053350 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:48.981165 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1366/2728] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:49.255298 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.256788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.268617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cf437-d818-4bdc-957d-b22453fcac4b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.274244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.274896 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.275182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.275461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] PATH_INFO: `/v3/projects/901b5f7243354f529ad1ed560b3703a3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.276674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901b5f7243354f529ad1ed560b3703a3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.278656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.327088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901b5f7243354f529ad1ed560b3703a3', 'request_id': 'req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:49.333443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901b5f7243354f529ad1ed560b3703a3', 'request_id': 'req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:49.339502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901b5f7243354f529ad1ed560b3703a3', 'request_id': 'req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:49.351752 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] The token cache is being invalidate because project 901b5f7243354f529ad1ed560b3703a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:52:49.352043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:49.352667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0beb928b-e98c-4db9-bf59-cc4ff28e4ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:52:49.354199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1363/2729] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:52:49 2022] DELETE /identity/v3/projects/901b5f7243354f529ad1ed560b3703a3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:49.359039 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.359323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.359595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.379740 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:49.380928 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.397375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:49.440122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1367/2730] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:49.449577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.449871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.450154 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.469794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:49.471008 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.479159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1364/2731] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:49.483337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.493396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc1a2d-d922-48f0-b608-538db7c6c356 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.497935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.498534 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.498815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.499089 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.499770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:52:49.500072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.501691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.513651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1368/2732] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:52:49.518221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.527469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33ed61-4a1a-464f-b2d2-5df2e2d8b235 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.532034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.532634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.532944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.533243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.533906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.535840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.562333 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1365/2733] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:52:49.567170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.577014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8600af-2efb-40c4-8e06-2ffc740fc20e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.632411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.633059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.633343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.633621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.634302 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.636220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.656904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1369/2734] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:52:49.662162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.672467 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c0bc-9d62-479a-941e-9fa4a95cbfe7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.677449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.678056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.678338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.678614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.679306 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.680888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.691091 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1366/2735] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.695068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.705124 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607269b-3bc5-4315-9409-c818fc6fbace admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.710571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.711169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.711454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.711732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.712421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.714041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f25872-dcd6-4114-b357-2733302adde9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.724360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1370/2736] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:52:49.728317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.738290 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece10c0-4757-4ea4-b0b8-27a863c03736 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.742831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.743486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.743866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.744231 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] PATH_INFO: `/v3/projects/9632b7ed64cb4e91a571d4a6e5aa2a6e/users/ae8ea760a47c4711a2267bc3b03f65d8/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.759391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9632b7ed64cb4e91a571d4a6e5aa2a6e, user_id=ae8ea760a47c4711a2267bc3b03f65d8, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.762089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.774900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1367/2737] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:52:49 2022] PUT /identity/v3/projects/9632b7ed64cb4e91a571d4a6e5aa2a6e/users/ae8ea760a47c4711a2267bc3b03f65d8/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:52:49.785592 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.785885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.786162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.813170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] MFA Rules not processed for user `ae8ea760a47c4711a2267bc3b03f65d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:52:49.814379 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.876911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1371/2738] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:52:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:52:49.888662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:49.890365 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:49.912207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:49.951299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a5d1e-b1b4-4f77-a7c6-50588871ea87 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:49.957112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4256769bf5b4e81a15b985039cdbd80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:49.957820 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:49.958105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:49.958380 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:49.966905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:49.968339 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:49.977159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1368/2739] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:52.889675 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:52:52.891086 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:52.911953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:52:52.964776 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1aee2-3914-4808-8f7a-ca27313935f2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:52:52.969850 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:52:52.970470 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:52:52.970755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:52:52.971027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:52:52.972838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:52:52.984473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:52:52.986095 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945b50-bbc4-42a4-9344-afee0ff7a431 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:52:52.996995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1372/2740] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:52:59.973934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=17 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:02.992437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=24 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:29.310935 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:29.373546 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a456df0-15a1-4cc0-b017-23ed3c8f11eb service glance] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:29.377987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:29.378622 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:29.378903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:29.379178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] PATH_INFO: `/v3/users/ae8ea760a47c4711a2267bc3b03f65d8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:29.380331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae8ea760a47c4711a2267bc3b03f65d8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:29.382811 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:29.429130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae8ea760a47c4711a2267bc3b03f65d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.436262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae8ea760a47c4711a2267bc3b03f65d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.447824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae8ea760a47c4711a2267bc3b03f65d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.454727 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae8ea760a47c4711a2267bc3b03f65d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.455351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae8ea760a47c4711a2267bc3b03f65d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.461173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1369/2741] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:53:29 2022] DELETE /identity/v3/users/ae8ea760a47c4711a2267bc3b03f65d8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:29.465660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:29.466169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:29.466337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:29.485381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:29.486608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:29.543072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1373/2742] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:29.553626 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:29.555014 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:29.610294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b7957-eea4-44c0-a49d-f398fced4c42 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:29.613971 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:29.614578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:29.615433 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:29.615711 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:29.624687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:29.626088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b26c3-6214-47f0-80d6-0cea30190be8 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:29.635529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1370/2743] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:53:29.901336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:29.902754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:29.916050 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e81a1-a0cb-4d60-8f5f-70844f8f098a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:29.920748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:29.921425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:29.921707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:29.921979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] PATH_INFO: `/v3/projects/9632b7ed64cb4e91a571d4a6e5aa2a6e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:29.923130 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9632b7ed64cb4e91a571d4a6e5aa2a6e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:29.925247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:29.972439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9632b7ed64cb4e91a571d4a6e5aa2a6e', 'request_id': 'req-78c5e116-62a1-4146-a8ef-f6959176d0a9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.978727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9632b7ed64cb4e91a571d4a6e5aa2a6e', 'request_id': 'req-78c5e116-62a1-4146-a8ef-f6959176d0a9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.984700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9632b7ed64cb4e91a571d4a6e5aa2a6e', 'request_id': 'req-78c5e116-62a1-4146-a8ef-f6959176d0a9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.997242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] The token cache is being invalidate because project 9632b7ed64cb4e91a571d4a6e5aa2a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:53:29.997519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.998148 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-78c5e116-62a1-4146-a8ef-f6959176d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:53:29.999643 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1374/2744] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:53:29 2022] DELETE /identity/v3/projects/9632b7ed64cb4e91a571d4a6e5aa2a6e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:53:30.004510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.004800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.005114 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.024691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:30.025919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:30.042827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:30.093280 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1371/2745] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:53:30.103409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.103906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.104421 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.127197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:30.128403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:30.137121 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1375/2746] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:30.141502 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.152178 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf6e40-d2b6-4100-b3c3-f6f1e3857a91 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.156684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.157344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.157620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.157892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.158578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:53:30.158877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.160446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.172299 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1372/2747] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:53:30.176478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.186194 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23182fac-1812-42f1-95ff-c2df171a0e60 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.190900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.191502 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.191781 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.192055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.192708 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.195434 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.222297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1376/2748] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:53:30 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:53:30.226358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.236166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a0d28-300f-4f55-ae79-a49c93c4c46f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.292245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.292858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.293167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.293446 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.294112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.296019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.316722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1373/2749] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:53:30 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:53:30.321617 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.331953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34024f-312f-4bbf-8934-331901819753 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.337488 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.338088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.338368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.338642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.339327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.340913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.351542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1377/2750] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:30.355273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.365005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c358ca2-3997-4870-8cfb-4746edd5db72 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.369823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.370426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.370710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.370986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.371795 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.373440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb369a5-d672-4094-ab84-8b15d876ba20 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.384613 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1374/2751] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:53:30.388572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.399182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e56ea-7d41-42b7-9537-26814a46db0d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.403987 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.404579 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.404861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.405177 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] PATH_INFO: `/v3/projects/57ef5c5bb40d4ae8b4b539777fa698c6/users/fe392d815d0a4520b3082caaac5daf27/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.419321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ef5c5bb40d4ae8b4b539777fa698c6, user_id=fe392d815d0a4520b3082caaac5daf27, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.422014 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.434569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1378/2752] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:53:30 2022] PUT /identity/v3/projects/57ef5c5bb40d4ae8b4b539777fa698c6/users/fe392d815d0a4520b3082caaac5daf27/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:53:30.445393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.445682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.445959 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.473980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] MFA Rules not processed for user `fe392d815d0a4520b3082caaac5daf27`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:53:30.475174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:30.539331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1375/2753] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:53:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:53:30.550543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:53:30.552810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:53:30.574416 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:30.610312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3b75-304c-47f5-bf26-42e9f03f54c1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:53:30.613921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:53:30.614525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:53:30.615370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:53:30.615804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:53:30.623973 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:53:30.625487 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:53:30.634976 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1379/2754] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:53:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:53:40.536472 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:53:40.630208 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=17 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:04.277017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:04.278440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:04.334371 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853a652-b5a0-4a4e-92ab-4c3eb4825f26 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:04.338496 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:04.339105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:04.339387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:04.339691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:04.341471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:04.402256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:04.403693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:04.413833 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1376/2755] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:07.557639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:07.559077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:07.573940 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f07d0-d303-41ef-8dea-2039941b7fce service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:07.578444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:07.579068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:07.579348 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:07.579913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] PATH_INFO: `/v3/users/30051341684845938161bb5ec0680d68` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:07.581135 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] RBAC: Authorizing `identity:delete_user(user_id=30051341684845938161bb5ec0680d68)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:07.583104 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:07.633026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30051341684845938161bb5ec0680d68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.644609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30051341684845938161bb5ec0680d68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.645275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30051341684845938161bb5ec0680d68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.652178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30051341684845938161bb5ec0680d68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.659510 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b0c1443c-c50c-4f4f-b585-aca8c2f313af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30051341684845938161bb5ec0680d68'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.666748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1380/2756] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:07 2022] DELETE /identity/v3/users/30051341684845938161bb5ec0680d68 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:07.844788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:07.855071 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b94ba-a792-4e85-87bf-b724b5b17768 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:07.915838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:07.916485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:07.916792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:07.917112 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] PATH_INFO: `/v3/projects/6a1fbeedf5134a5a9432e42a35e1d955` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:07.918272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1fbeedf5134a5a9432e42a35e1d955)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:07.920190 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:07.967210 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1fbeedf5134a5a9432e42a35e1d955', 'request_id': 'req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.969121 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:07.973589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1fbeedf5134a5a9432e42a35e1d955', 'request_id': 'req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.980398 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1fbeedf5134a5a9432e42a35e1d955', 'request_id': 'req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.996784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] The token cache is being invalidate because project 6a1fbeedf5134a5a9432e42a35e1d955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:07.997104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.997756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-51116d7e-418b-4add-9d9b-4e6a7f3b0a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:07.999277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1377/2757] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:07 2022] DELETE /identity/v3/projects/6a1fbeedf5134a5a9432e42a35e1d955 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:08.004034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.004318 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.004590 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.025411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:08.026623 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:08.086361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1381/2758] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:08.094420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.094663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.094939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.114689 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:08.115905 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:08.124490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1378/2759] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:08.129006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.139658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931a168-df29-437c-83cd-c34fa3cafaef None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.144746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.145396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.145677 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.145946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.146832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:08.147134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.149351 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.161364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1382/2760] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:08.166005 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.175166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e61be-d1d9-497a-98f1-dc974dccb572 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.180110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.180712 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.181031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.181313 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.181976 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.183944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.212577 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1379/2761] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:08 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:08.216884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.227828 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f6a0-e10b-4313-b475-00129caa397a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.286412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.287032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.287309 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.287582 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.288527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.290629 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.298381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:08.315883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1383/2762] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:08 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:08.317372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.333173 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec0a87-160a-474d-b563-de4cb06bd556 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.338817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.339437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.339714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.339985 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.340707 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.342340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.352655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1380/2763] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:08.356689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.366982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac528-4f49-451c-a2df-12c9947f37b6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.371659 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.372271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.372561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.372840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.373569 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.375171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdd845-307c-4018-93aa-bd95f8f84840 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.386694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1384/2764] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:08.390747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.401329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18072f-fe78-46b7-8dd7-563699f25e8c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.406170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.406828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.407105 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.407379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] PATH_INFO: `/v3/projects/789ff552e0064c67b4c72e80b4c2f07d/users/b65bbc5d35d345fb8c7b0eda50c9fca0/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.422651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=789ff552e0064c67b4c72e80b4c2f07d, user_id=b65bbc5d35d345fb8c7b0eda50c9fca0, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.425324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.437762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1381/2765] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:08 2022] PUT /identity/v3/projects/789ff552e0064c67b4c72e80b4c2f07d/users/b65bbc5d35d345fb8c7b0eda50c9fca0/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:08.442716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080075-db4c-4747-b0e2-df0ba44fb453 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.442995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080075-db4c-4747-b0e2-df0ba44fb453 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.443263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080075-db4c-4747-b0e2-df0ba44fb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.462817 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-32080075-db4c-4747-b0e2-df0ba44fb453 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:08.464005 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-32080075-db4c-4747-b0e2-df0ba44fb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:08.520650 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1385/2766] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:08.530964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:08.532654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:08.590244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898939-1c2a-4ba8-b214-029cf9ffdc5b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:08.594070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:08.594713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:08.594994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:08.595268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:08.605429 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:08.606802 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:08.616656 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1382/2767] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:09.235196 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.235716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.235815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.264097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] MFA Rules not processed for user `b65bbc5d35d345fb8c7b0eda50c9fca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:09.265365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:09.333327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1386/2768] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:09.338595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.348401 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c378c2-b0ed-44f4-9b45-83e23fce3663 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.353909 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.354554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.354839 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.355119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.355788 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.357783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.388554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1383/2769] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:09 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:09.392764 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.403941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f7b8d-e6c4-467a-91d2-96c88de09c4b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.458158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.458693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.458971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.459235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.460489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.462414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.478527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:09.485770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1387/2770] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:09 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:09.489736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.500250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5394-914f-4bb6-ab7b-9397c3c72c3a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.504870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.505503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.505787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.506064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.507274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.508999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.519835 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1384/2771] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:09.524826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.534563 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c617d-c61a-48ce-9185-5634ebadea49 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.539213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.539853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.540133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.540404 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] PATH_INFO: `/v3/projects/a396f2319c50405c9051a81471b56d83/users/4612663baaa943cd9e66447146271d25/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.554614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a396f2319c50405c9051a81471b56d83, user_id=4612663baaa943cd9e66447146271d25, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.557274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.570871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1388/2772] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:09 2022] PUT /identity/v3/projects/a396f2319c50405c9051a81471b56d83/users/4612663baaa943cd9e66447146271d25/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:09.575798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.586737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934d4b0-faba-487d-b65c-f41d5fa469c8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.629947 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:09.644221 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.644834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.645148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.645423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.646132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.648614 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6730e3c-9358-4597-aa81-95a62068589d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.658260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1385/2773] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:09.662461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:09.672931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc5b42-78d9-44c2-acc1-b5caff69e089 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:09.677516 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:09.678155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:09.678429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:09.678920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] PATH_INFO: `/v3/projects/a396f2319c50405c9051a81471b56d83/users/4612663baaa943cd9e66447146271d25/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:09.680842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a396f2319c50405c9051a81471b56d83, user_id=4612663baaa943cd9e66447146271d25, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:09.683478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:09.693352 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1389/2774] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:09 2022] PUT /identity/v3/projects/a396f2319c50405c9051a81471b56d83/users/4612663baaa943cd9e66447146271d25/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:10.318349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:10.318733 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:10.319148 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:10.350895 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] MFA Rules not processed for user `4612663baaa943cd9e66447146271d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:10.352226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:10.426367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1386/2775] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:11.415280 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.422024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:11.483980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.511945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac122a6-c2e9-49ae-a44d-9b3872900ad1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.518492 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.519112 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.519396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.519748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.521482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:11.536494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.538855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.549338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1390/2776] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:11.559471 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa050cd0-14f9-4648-bd09-07ea4d86c229 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.566739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.567565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.567852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.568135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] PATH_INFO: `/v3/users/b65bbc5d35d345fb8c7b0eda50c9fca0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.570942 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65bbc5d35d345fb8c7b0eda50c9fca0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.572890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.626552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65bbc5d35d345fb8c7b0eda50c9fca0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.634513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65bbc5d35d345fb8c7b0eda50c9fca0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.647072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:11.650709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65bbc5d35d345fb8c7b0eda50c9fca0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.660288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65bbc5d35d345fb8c7b0eda50c9fca0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.665236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65bbc5d35d345fb8c7b0eda50c9fca0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.671964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1387/2777] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:11 2022] DELETE /identity/v3/users/b65bbc5d35d345fb8c7b0eda50c9fca0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:11.852073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:11.862664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9defef-77df-4ae6-8366-37eaac924af4 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:11.923399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:11.924078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:11.924360 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:11.924638 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] PATH_INFO: `/v3/projects/789ff552e0064c67b4c72e80b4c2f07d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:11.925885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=789ff552e0064c67b4c72e80b4c2f07d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:11.927801 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:11.975322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789ff552e0064c67b4c72e80b4c2f07d', 'request_id': 'req-0c484b20-1545-4e64-829c-5af7ec088e0d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.978371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:11.982676 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789ff552e0064c67b4c72e80b4c2f07d', 'request_id': 'req-0c484b20-1545-4e64-829c-5af7ec088e0d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:11.990247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789ff552e0064c67b4c72e80b4c2f07d', 'request_id': 'req-0c484b20-1545-4e64-829c-5af7ec088e0d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.003453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] The token cache is being invalidate because project 789ff552e0064c67b4c72e80b4c2f07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:12.003745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.004464 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.006042 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1391/2778] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:11 2022] DELETE /identity/v3/projects/789ff552e0064c67b4c72e80b4c2f07d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:12.010185 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.011576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:12.024757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19312a-cfd4-4d77-a0a0-aab1dc0fbb5e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.083574 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.084250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.084570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.085053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] PATH_INFO: `/v3/users/4612663baaa943cd9e66447146271d25` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.086615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4612663baaa943cd9e66447146271d25)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.088575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.136415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4612663baaa943cd9e66447146271d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.143139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4612663baaa943cd9e66447146271d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.155332 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4612663baaa943cd9e66447146271d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.162017 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4612663baaa943cd9e66447146271d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.162645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b72bbc8e-292a-41ab-8cc1-3d839ae51144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4612663baaa943cd9e66447146271d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.168951 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1388/2779] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:12 2022] DELETE /identity/v3/users/4612663baaa943cd9e66447146271d25 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:12.352584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.354027 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:12.367138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c484b20-1545-4e64-829c-5af7ec088e0d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.424580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.425259 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.425514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.425791 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] PATH_INFO: `/v3/projects/a396f2319c50405c9051a81471b56d83` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.426941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a396f2319c50405c9051a81471b56d83)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.429859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.476088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a396f2319c50405c9051a81471b56d83', 'request_id': 'req-7c4d0f67-e592-4364-b56d-0689e30d0955'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.481821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a396f2319c50405c9051a81471b56d83', 'request_id': 'req-7c4d0f67-e592-4364-b56d-0689e30d0955'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.489787 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a396f2319c50405c9051a81471b56d83', 'request_id': 'req-7c4d0f67-e592-4364-b56d-0689e30d0955'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.502101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] The token cache is being invalidate because project a396f2319c50405c9051a81471b56d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:12.502438 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.503059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7c4d0f67-e592-4364-b56d-0689e30d0955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:12.504564 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1392/2780] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:12 2022] DELETE /identity/v3/projects/a396f2319c50405c9051a81471b56d83 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:12.509715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.510189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.510349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.530312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:12.531523 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:12.590596 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1389/2781] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:12.598350 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5379-0578-4621-854c-957198faed4d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.598642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5379-0578-4621-854c-957198faed4d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.598922 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f5379-0578-4621-854c-957198faed4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.619571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6a5f5379-0578-4621-854c-957198faed4d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:12.620763 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6a5f5379-0578-4621-854c-957198faed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:12.628839 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1393/2782] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:12.633033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.642425 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4ad8-f2f0-4226-9090-922f525878d2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.646958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.647541 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.647894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.648174 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.648861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:12.649307 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.650883 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.652291 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:12.665242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1390/2783] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:12.669325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f5379-0578-4621-854c-957198faed4d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.679461 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f5379-0578-4621-854c-957198faed4d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.684072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.684673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.684985 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.685260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.685923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.687808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.715423 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1394/2784] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:12 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:12.719464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.730016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e828b1d-d1a4-4032-b700-aae3415b42be admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.789303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.789929 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.790206 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.790560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.791126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.793048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.820658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1391/2785] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:12 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:12.821953 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.841202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac3d8-71f4-47df-8293-71e8de26f8dc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.841202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.841202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.841202 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.841538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.841538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.842435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.856349 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1395/2786] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:12.857667 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.868659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5dce2-f683-4cec-8e0a-7f766aff7dd5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.874007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.874631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.874916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.875194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.875887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.877586 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44048a-e75f-4ae7-93e0-3f2273c62786 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.889007 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1392/2787] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:12.893220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:12.904150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481672a2-e885-4c44-bf12-a29a3781645b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:12.909306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:12.910012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.910295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.910575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] PATH_INFO: `/v3/projects/7d767540e5ee4732a17826e94fa6e639/users/7f7649051fe249bdbfca57f9d82834e5/roles/cfc89a51451c403880880749d727c042` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.926727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d767540e5ee4732a17826e94fa6e639, user_id=7f7649051fe249bdbfca57f9d82834e5, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:12.929960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:12.949678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1396/2788] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:12 2022] PUT /identity/v3/projects/7d767540e5ee4732a17826e94fa6e639/users/7f7649051fe249bdbfca57f9d82834e5/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:12.957142 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:12.957448 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:12.957753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:12.985894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] MFA Rules not processed for user `7f7649051fe249bdbfca57f9d82834e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:12.987109 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.051485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1393/2789] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:13.056387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.065174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:13.124006 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a325ed-7a28-4034-b0cb-b3da422cbe7e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.129077 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.129610 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.130004 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.130258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.130999 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.133175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.159685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1397/2790] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:13 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:13.164634 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.175404 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaee5d-3f0f-497d-94bc-9eb5c1854a24 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.237260 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.237890 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.238170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.238444 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.239107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.241059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.249861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:13.266536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1394/2791] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:13 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:13.267765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.279106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd98b07-dd68-48b2-b59a-70f8ce2dc8b1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.284699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.285355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.285661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.285938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.286635 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.288455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.299326 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1398/2792] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:13.303458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.313706 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93f6ea-58e6-492f-a270-62d924f5042e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.318261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.318903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.319737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.320036 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] PATH_INFO: `/v3/projects/15b461e90877422580d369d6f542b098/users/d8e18b48262840b9af1d548b3cf89399/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.335684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b461e90877422580d369d6f542b098, user_id=d8e18b48262840b9af1d548b3cf89399, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.338391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.351642 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1395/2793] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:13 2022] PUT /identity/v3/projects/15b461e90877422580d369d6f542b098/users/d8e18b48262840b9af1d548b3cf89399/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:13.355612 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.365495 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3242d5-7209-43a5-ae03-34314abfcb8b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.420951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.421596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.421921 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.422237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.423012 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.424627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1dfc-b1d5-4659-a8a9-3a518402b831 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.434963 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1399/2794] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:13.439177 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.449479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789387b1-ca47-483a-a372-d35a03da26a2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.454048 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.454702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.455244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.455524 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] PATH_INFO: `/v3/projects/15b461e90877422580d369d6f542b098/users/d8e18b48262840b9af1d548b3cf89399/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.457537 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b461e90877422580d369d6f542b098, user_id=d8e18b48262840b9af1d548b3cf89399, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.460159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.470271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1396/2795] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:13 2022] PUT /identity/v3/projects/15b461e90877422580d369d6f542b098/users/d8e18b48262840b9af1d548b3cf89399/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:13.480753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.481068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.481345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.508281 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] MFA Rules not processed for user `d8e18b48262840b9af1d548b3cf89399`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:13.509535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.574707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1400/2796] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:13.586920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:13.588757 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:13.648481 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8cfac-b3df-4f61-bcf6-8fcc10d654d9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:13.652635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:13.653274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:13.653551 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:13.653821 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:13.662534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:13.663981 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5ccb9-c2b3-4746-9a36-efcf44df2183 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:13.673800 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1397/2797] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:14.058418 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.062962 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:14.135797 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5b8f-2122-46b8-be0d-363469827547 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.140335 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.140989 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.141278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.141556 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] PATH_INFO: `/v3/users/7f7649051fe249bdbfca57f9d82834e5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.142714 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7649051fe249bdbfca57f9d82834e5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.144636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.189565 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7649051fe249bdbfca57f9d82834e5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.201357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7649051fe249bdbfca57f9d82834e5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.202028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7649051fe249bdbfca57f9d82834e5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.209597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7649051fe249bdbfca57f9d82834e5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.217520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7649051fe249bdbfca57f9d82834e5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.223320 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1401/2798] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:14 2022] DELETE /identity/v3/users/7f7649051fe249bdbfca57f9d82834e5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:14.228406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.228698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.229002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.249209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:14.250393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.310473 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1398/2799] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:14.320452 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.322049 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.381964 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768b7d9-df37-4061-bd5e-8b5e835f4969 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.386026 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.386640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.386917 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.387189 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.396033 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.397509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.407699 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1402/2800] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:14.691585 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.693098 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:14.705843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e049963-3d71-4e66-97ab-d9954c51c780 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.710529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.711288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.712107 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.712384 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] PATH_INFO: `/v3/projects/7d767540e5ee4732a17826e94fa6e639` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.713670 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d767540e5ee4732a17826e94fa6e639)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.715592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.763101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d767540e5ee4732a17826e94fa6e639', 'request_id': 'req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.764919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:14.769650 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d767540e5ee4732a17826e94fa6e639', 'request_id': 'req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.777431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d767540e5ee4732a17826e94fa6e639', 'request_id': 'req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.789992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] The token cache is being invalidate because project 7d767540e5ee4732a17826e94fa6e639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:14.790273 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.790913 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.792428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1399/2801] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:14 2022] DELETE /identity/v3/projects/7d767540e5ee4732a17826e94fa6e639 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:14.796359 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:14.806215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed784-5d6f-4a03-adcd-bc9490367802 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:14.826789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:14.864871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:14.865533 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:14.865815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:14.866092 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] PATH_INFO: `/v3/users/d8e18b48262840b9af1d548b3cf89399` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:14.867242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e18b48262840b9af1d548b3cf89399)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:14.869477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:14.915363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e18b48262840b9af1d548b3cf89399'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.929616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e18b48262840b9af1d548b3cf89399'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.930783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e18b48262840b9af1d548b3cf89399'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.938364 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e18b48262840b9af1d548b3cf89399'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.944992 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b76aeb3-fbd4-42ec-94ce-33253c8dd0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e18b48262840b9af1d548b3cf89399'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:14.951441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1403/2802] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:14 2022] DELETE /identity/v3/users/d8e18b48262840b9af1d548b3cf89399 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:15.249946 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.251405 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.265370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99e9c4-bca6-4d50-acf3-bcb123e8e057 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.323426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.324003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.324285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.324560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] PATH_INFO: `/v3/projects/15b461e90877422580d369d6f542b098` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.325847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b461e90877422580d369d6f542b098)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.327746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.374303 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b461e90877422580d369d6f542b098', 'request_id': 'req-ef1bc473-5a3d-48b6-9eb9-6e7414295352'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:15.380483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b461e90877422580d369d6f542b098', 'request_id': 'req-ef1bc473-5a3d-48b6-9eb9-6e7414295352'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:15.387776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b461e90877422580d369d6f542b098', 'request_id': 'req-ef1bc473-5a3d-48b6-9eb9-6e7414295352'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:15.400581 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] The token cache is being invalidate because project 15b461e90877422580d369d6f542b098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:15.400871 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:15.401533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ef1bc473-5a3d-48b6-9eb9-6e7414295352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:15.403167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1400/2803] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:15 2022] DELETE /identity/v3/projects/15b461e90877422580d369d6f542b098 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:15.408263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.408553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.408832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.429178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:15.430785 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.493706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1404/2804] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:15.501565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.501852 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.502126 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.522511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:15.523763 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.532720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1401/2805] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:15.537213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.546851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae98f5-a412-4f35-bd34-088e3f706d2d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.551356 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.551956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.552238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.552512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.553226 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:15.553531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.555119 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.566324 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1405/2806] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:15.570511 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.581031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c0d79-4f23-4e00-9e4c-3f0b3a422a43 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.585569 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.586164 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.587140 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.587654 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.588318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.590279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.616589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1402/2807] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:15.621406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.631409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6727e11-8f31-481a-936a-a496b30a4821 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.660995 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:15.689278 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.689883 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.690160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.690429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.691083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.693015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.714573 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1406/2808] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:15.718721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.729131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cabe9-1ea0-4f60-8579-820ec2919944 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.733691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.734297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.734576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.734848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.735534 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.737166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.747073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1403/2809] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:15.751019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.761323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5f4db-47f0-4ea5-8419-e42a6776a5d7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.765887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.766476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.766755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.767024 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.767701 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.769306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8b277-1d6f-4a65-b354-c26471c8e8c6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.780334 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1407/2810] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:15.784261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.794321 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047715f3-84db-4357-a366-da1a457a682d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:15.798863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:15.799518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.799794 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.800067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] PATH_INFO: `/v3/projects/61de1be741c4405c9bcd2f5e58303029/users/04b800ce8b6e413381308be6e942e9d3/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.814104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61de1be741c4405c9bcd2f5e58303029, user_id=04b800ce8b6e413381308be6e942e9d3, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:15.816719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:15.830559 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1404/2811] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:15 2022] PUT /identity/v3/projects/61de1be741c4405c9bcd2f5e58303029/users/04b800ce8b6e413381308be6e942e9d3/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:15.842661 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:15.843134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:15.843134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:15.870062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] MFA Rules not processed for user `04b800ce8b6e413381308be6e942e9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:15.871247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:15.934143 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1408/2812] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:15.938615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:15.948064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.002955 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36c076-1545-4dd7-b052-49511e35d354 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.007818 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.009205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.009490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.009768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.010432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.012354 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.038130 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1405/2813] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:15 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:16.042742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.053559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1b879-fb6c-4e1a-8be3-bf454189ca23 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.110362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.110911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.111193 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.111472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.112132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.114099 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.135274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1409/2814] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:16 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:16.139329 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.149776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d482220-2add-4fbe-b829-570f180aa308 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.154341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.154944 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.155222 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.155498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.156181 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.157838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.168582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1406/2815] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:16.172482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.182926 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed710-77b8-4647-bd2e-f2028f2a7566 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.187570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.188217 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.188491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.188762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] PATH_INFO: `/v3/projects/ca829bc1747d4b83bc3f17637c552a91/users/b471b813c0a54ba18460a1ececdd8f99/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.204011 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca829bc1747d4b83bc3f17637c552a91, user_id=b471b813c0a54ba18460a1ececdd8f99, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.207019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.219561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1410/2816] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:16 2022] PUT /identity/v3/projects/ca829bc1747d4b83bc3f17637c552a91/users/b471b813c0a54ba18460a1ececdd8f99/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:16.223678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.234668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6ba53-6be4-4d3b-b11e-0bfc28b940af admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.252837 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.254600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.260871 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.303338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.303338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.303338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.303338 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.304166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.304166 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.317847 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1407/2817] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:16.318891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.329187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d52b58-cde6-4cc4-99e1-0c1399158a73 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.332509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cfd52-1447-40fe-a2d2-8a7bc2094e15 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.334327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.335169 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.335458 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.335784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] PATH_INFO: `/v3/projects/ca829bc1747d4b83bc3f17637c552a91/users/b471b813c0a54ba18460a1ececdd8f99/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.338172 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.338319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca829bc1747d4b83bc3f17637c552a91, user_id=b471b813c0a54ba18460a1ececdd8f99, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.338833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.339113 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.339390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] PATH_INFO: `/v3/users/fe392d815d0a4520b3082caaac5daf27` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.340609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe392d815d0a4520b3082caaac5daf27)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.341187 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174809f-ca6e-4042-a570-3340a6875ce7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.342553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.351671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1408/2818] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:16 2022] PUT /identity/v3/projects/ca829bc1747d4b83bc3f17637c552a91/users/b471b813c0a54ba18460a1ececdd8f99/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:16.362320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.362557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.362834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.392722 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] MFA Rules not processed for user `b471b813c0a54ba18460a1ececdd8f99`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:16.392935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe392d815d0a4520b3082caaac5daf27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.394035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.405491 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe392d815d0a4520b3082caaac5daf27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.406236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe392d815d0a4520b3082caaac5daf27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.413484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe392d815d0a4520b3082caaac5daf27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.422342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2e6d4566-7e77-457d-8bd9-662de91ca8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe392d815d0a4520b3082caaac5daf27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.428496 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1411/2819] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:16 2022] DELETE /identity/v3/users/fe392d815d0a4520b3082caaac5daf27 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:16.433385 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.433673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.433951 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.447089 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.452885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:16.454246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.467035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1409/2820] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:16.479443 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.480851 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.515220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1412/2821] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:16.525658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.527188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.538004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9bc42-4142-4dcc-9134-1f8a7de1fbb6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.554753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.588609 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4627d-55a6-46d3-b856-be9cb4083f73 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.588848 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.589500 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.589783 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.590059 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.592640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.593290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.593576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.593854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.603838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.605328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.614931 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1413/2822] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:54:16.651184 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.652633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89a0c5-a984-4213-8b68-500415f1024e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.662411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1410/2823] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:16.692032 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.693760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.707292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e4ff4-b780-4811-a9a9-6641d075f51f service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.713030 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.713789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.714103 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.714494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] PATH_INFO: `/v3/users/04b800ce8b6e413381308be6e942e9d3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.715890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b800ce8b6e413381308be6e942e9d3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.718064 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.771846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b800ce8b6e413381308be6e942e9d3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.788947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b800ce8b6e413381308be6e942e9d3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.790920 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b800ce8b6e413381308be6e942e9d3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.798762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b800ce8b6e413381308be6e942e9d3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.806748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b800ce8b6e413381308be6e942e9d3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.813028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1414/2824] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:16 2022] DELETE /identity/v3/users/04b800ce8b6e413381308be6e942e9d3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:16.817607 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.818106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.818196 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.838432 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:16.839657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.890263 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:16.898501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.900580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.907002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1411/2825] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:16.917162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:16.918543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:16.918761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cdce1-06de-47aa-b4e2-3e38b22d7bba admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.925372 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:16.925858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:16.926132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:16.926422 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] PATH_INFO: `/v3/projects/57ef5c5bb40d4ae8b4b539777fa698c6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:16.927794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ef5c5bb40d4ae8b4b539777fa698c6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:16.930197 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:16.980101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ef5c5bb40d4ae8b4b539777fa698c6', 'request_id': 'req-163531f6-4cd9-49ac-95a3-f8acc4cab29d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.982183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556b69-e485-46f4-b991-b3376a750b7a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:16.985924 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ef5c5bb40d4ae8b4b539777fa698c6', 'request_id': 'req-163531f6-4cd9-49ac-95a3-f8acc4cab29d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:16.992312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ef5c5bb40d4ae8b4b539777fa698c6', 'request_id': 'req-163531f6-4cd9-49ac-95a3-f8acc4cab29d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.004899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] The token cache is being invalidate because project 57ef5c5bb40d4ae8b4b539777fa698c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:17.005267 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.005910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-163531f6-4cd9-49ac-95a3-f8acc4cab29d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.007472 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1415/2826] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:16 2022] DELETE /identity/v3/projects/57ef5c5bb40d4ae8b4b539777fa698c6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:17.013306 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d438c-d682-4647-be35-5978e7a8915a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.013580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d438c-d682-4647-be35-5978e7a8915a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.013860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d438c-d682-4647-be35-5978e7a8915a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.033600 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-640d438c-d682-4647-be35-5978e7a8915a None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:17.034811 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-640d438c-d682-4647-be35-5978e7a8915a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.035565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.036183 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.036463 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.036741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.038479 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.051805 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:17.100859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1416/2827] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:17.106633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.108132 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.108664 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.108949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.109250 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.118193 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1412/2828] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:54:17.128361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:17.129574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.138235 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1417/2829] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:17.142587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.153256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ef542-d7ba-4be2-bbb8-b55e49e94812 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.157888 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.158498 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.158804 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.159075 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.159756 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:17.160055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.161669 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.173483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1413/2830] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:54:17.178169 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.188640 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736b34-1aef-4779-b91e-33fbb9d27250 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.193656 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.194258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.194599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.194802 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.195574 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.198096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.226051 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1418/2831] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:17 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:17.230430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.240620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428719d8-807d-4437-8ff6-9187f3f3e6e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.289001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:17.301826 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.302449 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.302731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.303003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.303663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.305671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.327462 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1414/2832] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:17 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:17.331907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.342840 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c124-160b-49ea-b6f2-38ec2b7dcb7a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.348603 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.348879 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.349198 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.349471 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.350166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.351761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.361685 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1419/2833] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:17.365701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.376256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6832541-9bd0-4434-b5f8-a3c5edea2110 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.381362 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.381975 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.382368 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.382646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.383494 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.385327 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.396037 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.401370 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1415/2834] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:17.401514 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.402573 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.413781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e60962-b6e5-4157-83e8-770c589f98a0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.420073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.420772 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.421459 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.421588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] PATH_INFO: `/v3/projects/c51423871031456a80da2d9d0c74bc62/users/060f4e9543a5456897d5217676cb93a6/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.431061 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6d218-6074-4ce9-bf5a-ca1645639489 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.436432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.437097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.437381 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.437653 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] PATH_INFO: `/v3/projects/61de1be741c4405c9bcd2f5e58303029` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.438741 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51423871031456a80da2d9d0c74bc62, user_id=060f4e9543a5456897d5217676cb93a6, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.438861 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61de1be741c4405c9bcd2f5e58303029)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.440693 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.441640 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27005b36-aadd-4af7-88b5-893110f7bdbc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.455218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1416/2835] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:17 2022] PUT /identity/v3/projects/c51423871031456a80da2d9d0c74bc62/users/060f4e9543a5456897d5217676cb93a6/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:17.459984 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.460269 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.460542 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.482827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:17.484157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.492466 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61de1be741c4405c9bcd2f5e58303029', 'request_id': 'req-59c922c8-e270-4173-be37-dbdbe11b5a69'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.498694 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61de1be741c4405c9bcd2f5e58303029', 'request_id': 'req-59c922c8-e270-4173-be37-dbdbe11b5a69'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.505117 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61de1be741c4405c9bcd2f5e58303029', 'request_id': 'req-59c922c8-e270-4173-be37-dbdbe11b5a69'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.517535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] The token cache is being invalidate because project 61de1be741c4405c9bcd2f5e58303029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:17.517821 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.518465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.520166 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1420/2836] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:17 2022] DELETE /identity/v3/projects/61de1be741c4405c9bcd2f5e58303029 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:17.524774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.526369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.546697 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1417/2837] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:17.556461 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:17.557879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:17.593258 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c922c8-e270-4173-be37-dbdbe11b5a69 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.598778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.599426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.599710 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.599986 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] PATH_INFO: `/v3/users/b471b813c0a54ba18460a1ececdd8f99` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.601245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b471b813c0a54ba18460a1ececdd8f99)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.603275 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.610028 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:17.626544 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1802fe-23b7-43fa-b0d5-279497c0a7bb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:17.630499 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:17.631270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:17.631651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:17.633253 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:17.646801 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:17.648316 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:17.656669 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b471b813c0a54ba18460a1ececdd8f99'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.659510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1418/2838] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:54:17.668885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b471b813c0a54ba18460a1ececdd8f99'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.669831 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b471b813c0a54ba18460a1ececdd8f99'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.676957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b471b813c0a54ba18460a1ececdd8f99'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.688329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8b9660b-65ef-497c-9915-1a5f51d899a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b471b813c0a54ba18460a1ececdd8f99'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:17.694581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1421/2839] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:17 2022] DELETE /identity/v3/users/b471b813c0a54ba18460a1ececdd8f99 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:18.083628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.099200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834871c-a828-4164-99cf-11c28d617266 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.167939 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:18.212566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.213528 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.213910 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.214250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] PATH_INFO: `/v3/projects/ca829bc1747d4b83bc3f17637c552a91` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.216375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca829bc1747d4b83bc3f17637c552a91)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.223100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.292265 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca829bc1747d4b83bc3f17637c552a91', 'request_id': 'req-1ab9877a-3759-4709-a0cc-811a5ca59984'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:18.298606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca829bc1747d4b83bc3f17637c552a91', 'request_id': 'req-1ab9877a-3759-4709-a0cc-811a5ca59984'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:18.307031 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca829bc1747d4b83bc3f17637c552a91', 'request_id': 'req-1ab9877a-3759-4709-a0cc-811a5ca59984'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:18.321428 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] The token cache is being invalidate because project ca829bc1747d4b83bc3f17637c552a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:18.321979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:18.322949 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1ab9877a-3759-4709-a0cc-811a5ca59984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:18.329228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1419/2840] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:18 2022] DELETE /identity/v3/projects/ca829bc1747d4b83bc3f17637c552a91 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:18.332358 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.333139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.333139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.359562 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:18.360784 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:18.437745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1422/2841] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:18.445378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.445715 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.446628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.473532 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:18.474858 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:18.497567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1420/2842] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:18.502460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.518626 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8256b46-e1be-42b5-9026-4c8c69b14fde None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.523862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.524627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.525145 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.525490 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.526317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:18.526679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.528557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.546347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1423/2843] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:18.550964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.567391 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fdccf-c61e-4c2b-8e09-02f7a01dfd79 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.574682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.575784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.577032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.577552 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.578474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.581224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.622138 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1421/2844] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:18 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:18.624370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.637990 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98163cf-097d-4f7a-b583-687f4d40d5f5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.717365 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.717904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.718188 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.718465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.719234 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.721518 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.751589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1424/2845] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:18 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:18.755655 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.768372 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e017-72cc-4a34-b404-ad38898069ef admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.774579 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.775823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.775823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.776012 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.776720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.778426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.798436 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1422/2846] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:18.801132 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.808727 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:18.814439 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552e504-3423-4c8c-83bf-e364fb129128 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.822020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.822658 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.823504 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.823960 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.824719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.826479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbab6a-9559-4613-aa7e-b92cd7adec47 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.843566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1425/2847] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:18.847508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:18.859991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e498d-70dd-4136-807b-329f3d71b249 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:18.864999 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:18.865671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.865954 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.866223 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] PATH_INFO: `/v3/projects/9e3304f78c794619be72de13f09f33fb/users/1ad2075d7ea24684a7d0ac802c8c0d95/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.882994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3304f78c794619be72de13f09f33fb, user_id=1ad2075d7ea24684a7d0ac802c8c0d95, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:18.885958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:18.901294 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1423/2848] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:18 2022] PUT /identity/v3/projects/9e3304f78c794619be72de13f09f33fb/users/1ad2075d7ea24684a7d0ac802c8c0d95/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:18.913599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:18.913884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:18.914162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:18.944093 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] MFA Rules not processed for user `1ad2075d7ea24684a7d0ac802c8c0d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:18.945377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:19.021540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1426/2849] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:19.026860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.111986 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67e66c-67d3-4b0d-9dd7-0af0746e4f83 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.116940 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.118382 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.118510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.118853 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.119617 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.121673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.157934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:19.167720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1424/2850] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:19 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:19.171478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.183337 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e74a2a-aa81-46bf-9816-0c325b9f5dc2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.268233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.268958 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.269331 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.269670 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.270649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.273094 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.304097 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1427/2851] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:19 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:19.306674 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.317281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe9016-d3db-4ce7-a721-8a03e4386b24 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.321990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.322605 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.323033 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.323194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.324318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.326055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.341011 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1425/2852] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:19.343357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.354479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd8229-2cac-4ecf-bb6f-7647841e1e6a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.359604 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.360288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.360571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.360846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] PATH_INFO: `/v3/projects/53ffd200b9684e61990674466b348525/users/211674e0ee7944d2863eab3e696b9dea/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.377792 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ffd200b9684e61990674466b348525, user_id=211674e0ee7944d2863eab3e696b9dea, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.380444 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.426867 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1428/2853] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:19 2022] PUT /identity/v3/projects/53ffd200b9684e61990674466b348525/users/211674e0ee7944d2863eab3e696b9dea/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:19.429750 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.440719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fd9a-56f7-4ec8-afde-883c3d0ee08a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.516419 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.517205 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.517623 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.517987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.518764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.520455 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f0d3d-f72c-41f5-8ec0-53b58d81c68c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.531992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1426/2854] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:19.536429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.548853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fd8a-2985-49f8-8484-99c03c2ba062 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.554120 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.554919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.555505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.555907 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] PATH_INFO: `/v3/projects/53ffd200b9684e61990674466b348525/users/211674e0ee7944d2863eab3e696b9dea/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.558118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ffd200b9684e61990674466b348525, user_id=211674e0ee7944d2863eab3e696b9dea, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.561140 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.572627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1429/2855] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:19 2022] PUT /identity/v3/projects/53ffd200b9684e61990674466b348525/users/211674e0ee7944d2863eab3e696b9dea/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:19.583331 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.583616 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.583885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.611807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] MFA Rules not processed for user `211674e0ee7944d2863eab3e696b9dea`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:19.613053 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:19.686226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1427/2856] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:19.699109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:19.700946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:19.742923 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:19.767015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149db1-5440-483e-992a-77fab6f593aa admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:19.771000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:19.771732 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:19.772016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:19.772294 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:19.782410 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:19.784155 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:19.794393 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1430/2857] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:20.202495 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.215646 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356709c5-8e2d-4547-addd-a91ff6a3cf1b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:20.223891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '211674e0ee7944d2863eab3e696b9dea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ffd200b9684e61990674466b348525', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:20.224725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:20.225200 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:20.225589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:20.226416 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:20.227078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:20.229517 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:20.250991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1428/2858] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:20.252055 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.271686 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b155c-84e2-408b-8412-4944506df4b2 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:20.281515 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '211674e0ee7944d2863eab3e696b9dea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ffd200b9684e61990674466b348525', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:20.282548 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:20.282833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:20.283116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:20.283804 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:20.293628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:20.340223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1431/2859] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:20 2022] POST /identity/v3/projects => generated 352 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:20.344387 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.355749 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e1eb-e4ba-46dd-b84a-a127d3901807 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:20.444343 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:20.457120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '211674e0ee7944d2863eab3e696b9dea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ffd200b9684e61990674466b348525', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:20.457891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:20.458705 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:20.459087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:20.460786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:20.461224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:20.464001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:20.483256 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1429/2860] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:54:20.518761 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.532614 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498f826-0e9b-4bcb-b664-fe58255bcd09 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:20.541657 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '211674e0ee7944d2863eab3e696b9dea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ffd200b9684e61990674466b348525', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:20.542690 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:20.543123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:20.543567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] PATH_INFO: `/v3/projects/c4a8b649fd054a3693eb43e8de29fbf0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:20.545571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c4a8b649fd054a3693eb43e8de29fbf0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:20.548160 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:20.613503 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a8b649fd054a3693eb43e8de29fbf0', 'request_id': 'req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:20.621137 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a8b649fd054a3693eb43e8de29fbf0', 'request_id': 'req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:20.629808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a8b649fd054a3693eb43e8de29fbf0', 'request_id': 'req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:20.645712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] The token cache is being invalidate because project c4a8b649fd054a3693eb43e8de29fbf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:20.646371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:20.647292 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc59184-eb1a-41c5-bbfd-7ef29a151e50 tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:20.653957 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1432/2861] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:20 2022] DELETE /identity/v3/projects/c4a8b649fd054a3693eb43e8de29fbf0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:20.941104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:20.943386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.034110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2636f4-39e5-4ad3-b2a4-f9af89ee1e6d tempest-VolumeQuotaClassesTest-1158554360 tempest-VolumeQuotaClassesTest-1158554360-project-admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:21.127293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:21.128297 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.128790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.129351 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] PATH_INFO: `/v3/users/1ad2075d7ea24684a7d0ac802c8c0d95` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.140476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad2075d7ea24684a7d0ac802c8c0d95)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:21.143440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.229420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad2075d7ea24684a7d0ac802c8c0d95'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.241234 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad2075d7ea24684a7d0ac802c8c0d95'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.256279 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad2075d7ea24684a7d0ac802c8c0d95'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.265344 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad2075d7ea24684a7d0ac802c8c0d95'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.265906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad2075d7ea24684a7d0ac802c8c0d95'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.273522 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1430/2862] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:20 2022] DELETE /identity/v3/users/1ad2075d7ea24684a7d0ac802c8c0d95 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:21.279152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.279152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.279152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.299271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:21.300500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.384457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1433/2863] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:21.392302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:21.393906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.430784 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:21.491469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878bbd-73c1-454b-b849-9529e4865bc7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:21.498501 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:21.499156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.499441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.499720 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.510352 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:21.511822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.533217 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1431/2864] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:21.887678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:21.890913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:21.914271 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d6fef-2904-4538-b3b8-959fc4e0a516 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:21.922297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:21.923105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:21.923505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:21.923869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] PATH_INFO: `/v3/projects/9e3304f78c794619be72de13f09f33fb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:21.925338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3304f78c794619be72de13f09f33fb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:21.928542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:21.972762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:21.987682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3304f78c794619be72de13f09f33fb', 'request_id': 'req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:21.995449 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3304f78c794619be72de13f09f33fb', 'request_id': 'req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.002783 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3304f78c794619be72de13f09f33fb', 'request_id': 'req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.019417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] The token cache is being invalidate because project 9e3304f78c794619be72de13f09f33fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:22.020108 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.020363 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0083fc36-cd2d-4b92-bae1-01ef31ecd6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.025720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1434/2865] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:21 2022] DELETE /identity/v3/projects/9e3304f78c794619be72de13f09f33fb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:22.026973 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.037740 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811633b1-065d-464c-b699-298023fc41cf service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.121504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.122293 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.122644 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.123020 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] PATH_INFO: `/v3/users/211674e0ee7944d2863eab3e696b9dea` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.125768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211674e0ee7944d2863eab3e696b9dea)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.127898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.193694 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211674e0ee7944d2863eab3e696b9dea'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.205994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211674e0ee7944d2863eab3e696b9dea'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.226507 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211674e0ee7944d2863eab3e696b9dea'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.249765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211674e0ee7944d2863eab3e696b9dea'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.250989 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211674e0ee7944d2863eab3e696b9dea'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.262156 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1432/2866] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:22 2022] DELETE /identity/v3/users/211674e0ee7944d2863eab3e696b9dea => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:22.345885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084262d6-a634-47f9-869c-11368a44f69f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.346171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084262d6-a634-47f9-869c-11368a44f69f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.346457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084262d6-a634-47f9-869c-11368a44f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.390307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-084262d6-a634-47f9-869c-11368a44f69f None None] MFA Rules not processed for user `060f4e9543a5456897d5217676cb93a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.391808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-084262d6-a634-47f9-869c-11368a44f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.494572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1435/2867] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:22.498849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.500281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.557998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:22.598781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e190-2dda-4ee2-9b0b-9a6bdd4732a4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.607938 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.608798 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.609113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.609389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.610078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.612504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.652199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1433/2868] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:22.658174 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084262d6-a634-47f9-869c-11368a44f69f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.672183 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084262d6-a634-47f9-869c-11368a44f69f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.679356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.682488 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.753240 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.754109 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.754970 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.755123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.756062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.758022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.767867 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d863-28b1-4deb-8f33-fa2df57498b6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.772609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.773286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.773704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.774106 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] PATH_INFO: `/v3/projects/53ffd200b9684e61990674466b348525` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.776032 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ffd200b9684e61990674466b348525)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.778420 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.788382 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1436/2869] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:22.790151 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.802935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c221e4-86da-4c72-bfa7-6d6f0826ef4e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.810068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.810765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.811021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.811299 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.812000 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.813642 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.828895 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1437/2870] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:22.831749 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.841990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ffd200b9684e61990674466b348525', 'request_id': 'req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.843628 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5390-c92b-4d9e-8ecb-93ef769039b1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:22.849673 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ffd200b9684e61990674466b348525', 'request_id': 'req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.857073 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ffd200b9684e61990674466b348525', 'request_id': 'req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.873241 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] The token cache is being invalidate because project 53ffd200b9684e61990674466b348525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:22.873557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.874314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d19b16d-430b-4ef0-bdf4-5f196bb98b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:22.875613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:22.877087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1434/2871] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:22 2022] DELETE /identity/v3/projects/53ffd200b9684e61990674466b348525 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:22.882159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8ef05-0959-49c3-83eb-60a44eb62c65 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.882739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8ef05-0959-49c3-83eb-60a44eb62c65 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.882739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8ef05-0959-49c3-83eb-60a44eb62c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.907591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8fa8ef05-0959-49c3-83eb-60a44eb62c65 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.909367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:22.909817 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8fa8ef05-0959-49c3-83eb-60a44eb62c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.910044 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.910323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.910599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] PATH_INFO: `/v3/projects/304318c87f724855bbccde4c0fc414e0/users/51830aea6d6040f6b0b09e13ea95dfa4/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.919199 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1435/2872] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:22.925849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=304318c87f724855bbccde4c0fc414e0, user_id=51830aea6d6040f6b0b09e13ea95dfa4, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:22.927990 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:22.928285 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:22.928572 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:22.928664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:22.944314 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1438/2873] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:22 2022] PUT /identity/v3/projects/304318c87f724855bbccde4c0fc414e0/users/51830aea6d6040f6b0b09e13ea95dfa4/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:22.948815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:22.950566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:22.953870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:22.955337 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:23.001664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:23.019397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1120ac-0fff-4acb-a990-b28e5d51ffaf admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.020454 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1436/2874] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:23.024476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.025173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.025282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.025453 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.025725 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.026414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.028022 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.035039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5bb55-dc86-4a42-a831-cf12b58ae4ed None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.040515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.040859 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1439/2875] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:23.044526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.044815 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.044913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.045125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.045968 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:23.046277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.047899 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.056341 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62277eae-f369-46f8-bb78-9326e678d6c6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.061150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.065001 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1437/2876] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:54:23.065156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.065156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.065156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] PATH_INFO: `/v3/projects/304318c87f724855bbccde4c0fc414e0/users/51830aea6d6040f6b0b09e13ea95dfa4/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.065156 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=304318c87f724855bbccde4c0fc414e0, user_id=51830aea6d6040f6b0b09e13ea95dfa4, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.065923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.067339 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.085411 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ecf14-711d-4c82-9e6c-71e7a26f6fdd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.085909 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1440/2877] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:23 2022] PUT /identity/v3/projects/304318c87f724855bbccde4c0fc414e0/users/51830aea6d6040f6b0b09e13ea95dfa4/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:23.153469 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.154097 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.154535 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.154695 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.155312 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.157271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.188019 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1438/2878] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:23 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:23.192403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.206457 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddeb61-bc95-4205-8707-6d6bbbc692cb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.286595 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.287621 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.287911 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.288199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.288875 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.291571 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.317069 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1441/2879] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:23 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:23.322249 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.334034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f15a-1d5b-43e4-83aa-036cfc71da59 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.339980 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.340789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.341110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.341388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.342119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.343775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.357863 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1439/2880] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:23.358691 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.369435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39576-930e-4e21-8745-671dfe4e6fb5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.374279 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.375013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.375390 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.375754 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.376549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.378398 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05adbe-b46d-407c-b04c-506eb6c8a7dd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.389893 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1442/2881] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:23.394356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.404592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a8eaa-cc41-4668-89da-15c1ffdef4ce admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.409850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.412267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.412267 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.412627 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] PATH_INFO: `/v3/projects/9d70a1de655d4434b91d19234ead13b2/users/6e171f038096418dac0ec7d02200ee6f/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.436226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d70a1de655d4434b91d19234ead13b2, user_id=6e171f038096418dac0ec7d02200ee6f, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.438834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.456314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1440/2882] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:23 2022] PUT /identity/v3/projects/9d70a1de655d4434b91d19234ead13b2/users/6e171f038096418dac0ec7d02200ee6f/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:23.467406 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.467654 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.467941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.493778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:23.497019 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] MFA Rules not processed for user `6e171f038096418dac0ec7d02200ee6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:23.499136 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:23.572493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1443/2883] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:23.577624 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.656119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652a40d-04df-4d85-8ee7-5701e6be806e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.662713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.664139 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.664508 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.665595 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.666342 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.668746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.700003 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1441/2884] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:23 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:23.704067 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.717371 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85cd50-0778-4304-abdc-bafaf7b78cff None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.788593 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.788884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.789178 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.790214 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.791186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.793512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.824290 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1444/2885] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:23 2022] POST /identity/v3/users => generated 302 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:23.828271 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.841358 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449805b-33ac-47f8-a415-f96e7ac342d9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.848278 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.849202 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.849493 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.849775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.850734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.853386 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.856558 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:23.870423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1442/2886] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 20:54:23.874068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.886236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43204d9b-5c36-4739-8d8d-b9fb416e14dc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:23.891096 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:23.891770 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:23.892056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:23.892336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] PATH_INFO: `/v3/projects/75284d40ab824a579066beb28f69948c/users/ed2cbea9f073442cb2b14c23ff6e5a04/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:23.907897 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] RBAC: Authorizing `identity:create_grant(project_id=75284d40ab824a579066beb28f69948c, user_id=ed2cbea9f073442cb2b14c23ff6e5a04, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:23.910541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:23.930688 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1445/2887] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:23 2022] PUT /identity/v3/projects/75284d40ab824a579066beb28f69948c/users/ed2cbea9f073442cb2b14c23ff6e5a04/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:23.932325 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:23.943078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dcfa6-2ac0-4297-935b-12466ce03f18 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:24.016167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:24.016807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.017123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.017403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.018104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:24.020317 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720c0ed-b2ab-4473-bb4f-11cd78a8cb59 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:24.033056 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1443/2888] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:24.035152 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:24.050237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f2375-0e2e-498a-9f82-336b424b782e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:24.056223 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:24.056910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.057218 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.057494 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] PATH_INFO: `/v3/projects/75284d40ab824a579066beb28f69948c/users/ed2cbea9f073442cb2b14c23ff6e5a04/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.060073 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75284d40ab824a579066beb28f69948c, user_id=ed2cbea9f073442cb2b14c23ff6e5a04, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:24.063106 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:24.082602 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1446/2889] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:24 2022] PUT /identity/v3/projects/75284d40ab824a579066beb28f69948c/users/ed2cbea9f073442cb2b14c23ff6e5a04/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:24.094251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.094538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.094816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.126016 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] MFA Rules not processed for user `ed2cbea9f073442cb2b14c23ff6e5a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:24.127244 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:24.212295 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1444/2890] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:24.221597 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:24.224087 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:24.320566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21325ca1-fc2c-4914-b8b1-94ba8c52b9d4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:24.325114 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:24.325786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.326110 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.326392 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.337455 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:24.338919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ecdc9-22ba-4513-b36b-611f9b338937 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:24.343277 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:24.353815 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1447/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:24.402719 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:24.405225 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:24.418898 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8436da-e343-456b-87f0-8fce3ece65da None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:24.422704 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:24.423491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:24.423875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:24.424251 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:24.493753 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:24.495182 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:24.506038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1445/2892] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:28.201327 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.201849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.201849 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.235054 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] MFA Rules not processed for user `51830aea6d6040f6b0b09e13ea95dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:28.236323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.324427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1448/2893] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:28.342054 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.343606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.372141 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:28.437979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f056eb-c4f9-414c-95a9-73148ce14b6e service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.443742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.444381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.444662 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.445021 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.447742 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.525261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.526872 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6147-80e6-411f-8038-2f7f0e635f20 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.537100 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1446/2894] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 20:54:28.718949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:28.799540 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37d243-a3f9-45fc-b91f-3157e48c6780 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:28.804796 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:28.805555 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.805856 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.806237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] PATH_INFO: `/v3/users/6e171f038096418dac0ec7d02200ee6f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.807546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e171f038096418dac0ec7d02200ee6f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:28.809478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:28.860171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e171f038096418dac0ec7d02200ee6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.873520 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e171f038096418dac0ec7d02200ee6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.874342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e171f038096418dac0ec7d02200ee6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.881832 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e171f038096418dac0ec7d02200ee6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.891721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e171f038096418dac0ec7d02200ee6f'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:28.897844 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1449/2895] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:28 2022] DELETE /identity/v3/users/6e171f038096418dac0ec7d02200ee6f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:28.902906 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:28.903192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:28.903466 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:28.923503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:28.924440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:28.992039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1447/2896] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:29.003625 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.005367 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:29.012767 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:29.075608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98d9a9-32ca-4886-8f59-30d1819609f7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.080755 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.081435 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.081719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.082111 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.105808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.107370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.118232 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1450/2897] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:29.507806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.510603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:29.527822 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9ecd-c37e-49a6-8fe7-b8832b2042bf None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.533035 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.533849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.534311 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.534725 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] PATH_INFO: `/v3/projects/9d70a1de655d4434b91d19234ead13b2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.536131 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d70a1de655d4434b91d19234ead13b2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.538497 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.597309 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d70a1de655d4434b91d19234ead13b2', 'request_id': 'req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.598961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:29.615685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d70a1de655d4434b91d19234ead13b2', 'request_id': 'req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.625505 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d70a1de655d4434b91d19234ead13b2', 'request_id': 'req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.643465 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] The token cache is being invalidate because project 9d70a1de655d4434b91d19234ead13b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:29.643831 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.644538 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.646761 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1448/2898] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:29 2022] DELETE /identity/v3/projects/9d70a1de655d4434b91d19234ead13b2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:29.651387 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:29.661718 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187810-c079-4b24-819f-d1df24526499 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:29.726679 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:29.728230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:29.728692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:29.729150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] PATH_INFO: `/v3/users/ed2cbea9f073442cb2b14c23ff6e5a04` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:29.730542 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2cbea9f073442cb2b14c23ff6e5a04)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:29.732668 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:29.791319 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2cbea9f073442cb2b14c23ff6e5a04'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.807747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2cbea9f073442cb2b14c23ff6e5a04'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.808484 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2cbea9f073442cb2b14c23ff6e5a04'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.816935 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2cbea9f073442cb2b14c23ff6e5a04'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.826397 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9696427b-e750-417e-a634-6274dbd1c5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2cbea9f073442cb2b14c23ff6e5a04'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:29.838414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1451/2899] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:29 2022] DELETE /identity/v3/users/ed2cbea9f073442cb2b14c23ff6e5a04 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:30.170483 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.171961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:30.187578 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2bfc8-cbcd-48ce-a2e1-e186ebd7e514 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.275776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.277195 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.277491 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.277775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] PATH_INFO: `/v3/projects/75284d40ab824a579066beb28f69948c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.279982 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=75284d40ab824a579066beb28f69948c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.282841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.363518 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75284d40ab824a579066beb28f69948c', 'request_id': 'req-9502d253-31c9-4133-81d2-574d76ccd55f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:30.372218 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75284d40ab824a579066beb28f69948c', 'request_id': 'req-9502d253-31c9-4133-81d2-574d76ccd55f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:30.379588 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75284d40ab824a579066beb28f69948c', 'request_id': 'req-9502d253-31c9-4133-81d2-574d76ccd55f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:30.395255 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] The token cache is being invalidate because project 75284d40ab824a579066beb28f69948c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:30.395556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:30.396201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9502d253-31c9-4133-81d2-574d76ccd55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:30.398046 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1449/2900] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:30 2022] DELETE /identity/v3/projects/75284d40ab824a579066beb28f69948c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:30.403343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.403636 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.403941 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.425778 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:30.427036 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:30.450445 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:30.500429 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1452/2901] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:30.509179 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.509698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.509775 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.535765 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:30.537061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:30.558576 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.558732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1450/2902] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:30.570323 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0a29-e69b-4bc7-996c-b203f43a238e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.576396 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.577078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.577403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.577647 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.581144 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:30.581463 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.583122 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.601596 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1453/2903] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 20:54:30.605916 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.616738 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe82c8-233e-4e92-9f93-1675a72b8415 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.622736 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.623366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.623647 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.623923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.625074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.626553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.659790 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1451/2904] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:30 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:30.660723 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.671729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab42f-c19b-4e5b-b3d4-dbc4ea1c8fdd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.740598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.741216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.741476 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.741748 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.742441 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.745501 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.772312 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1454/2905] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:30 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:30.774768 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.789167 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ef61-fb9f-40a1-9333-053c943cdef5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.794747 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.795780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.795780 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.795988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.796777 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.799224 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.817262 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1452/2906] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:30.818812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.835851 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3cd42-7062-4ac8-8147-afa44e097bac admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.843819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.844765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.845158 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.845500 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.846328 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.850100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2009c0-c77a-4893-a750-b272bb59c27a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.862965 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1455/2907] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:30.867194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:30.878373 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b970dd-766d-4cc9-a424-467854c08822 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:30.885094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:30.886318 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.886659 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.886873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] PATH_INFO: `/v3/projects/909231165dcb42dcbdc397f7447945a5/users/978781e3bd4b4b6bb9bb2814d3992600/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.905645 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=909231165dcb42dcbdc397f7447945a5, user_id=978781e3bd4b4b6bb9bb2814d3992600, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:30.908582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:30.928055 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1453/2908] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:30 2022] PUT /identity/v3/projects/909231165dcb42dcbdc397f7447945a5/users/978781e3bd4b4b6bb9bb2814d3992600/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:30.938297 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:30.938584 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:30.938912 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:30.974689 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] MFA Rules not processed for user `978781e3bd4b4b6bb9bb2814d3992600`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:30.975938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:31.058887 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1456/2909] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:31.063492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.076823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:31.151814 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f735-43d1-4782-aed8-9260125621bb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.159212 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.159879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.160162 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.160439 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.161149 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.163120 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.193145 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1454/2910] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:31 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:31.195425 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.207588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22daf7-e1f8-4069-9c6d-f50db2c09a39 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.224891 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:31.277072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.277702 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.277982 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.278255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.278925 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.280864 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.309477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1457/2911] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:31 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:31.310680 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.321731 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97a36c-2f59-4bca-8c83-f4e1384fe582 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.326319 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.326925 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.327209 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.327526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.328171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.329823 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.347703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1455/2912] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:31.352775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.363904 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94fed-0581-4a22-9d9e-38c9752e33b8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.371116 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.371721 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.372171 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.372345 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] PATH_INFO: `/v3/projects/dac4e8e33fec464aa578a6cfb110781d/users/9ef8c1d283ee41af9c1fa0ae64030f1b/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.433913 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac4e8e33fec464aa578a6cfb110781d, user_id=9ef8c1d283ee41af9c1fa0ae64030f1b, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.437846 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.454937 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1458/2913] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:31 2022] PUT /identity/v3/projects/dac4e8e33fec464aa578a6cfb110781d/users/9ef8c1d283ee41af9c1fa0ae64030f1b/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:31.459357 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.475766 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e73c2-4d03-4c1a-9584-f63a6cf774ff admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.561570 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.562418 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.562702 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.562974 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.563687 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.565378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7dbe6-6305-49d9-989d-e05c8386fe36 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.577022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1456/2914] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:31.581446 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.593288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf787d-ae52-437a-96e3-5cae4ce617a8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.598215 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.598892 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.599173 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.599473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] PATH_INFO: `/v3/projects/dac4e8e33fec464aa578a6cfb110781d/users/9ef8c1d283ee41af9c1fa0ae64030f1b/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.601498 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac4e8e33fec464aa578a6cfb110781d, user_id=9ef8c1d283ee41af9c1fa0ae64030f1b, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.604834 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.617246 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1459/2915] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:31 2022] PUT /identity/v3/projects/dac4e8e33fec464aa578a6cfb110781d/users/9ef8c1d283ee41af9c1fa0ae64030f1b/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:31.628789 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.629113 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.629394 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.662349 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] MFA Rules not processed for user `9ef8c1d283ee41af9c1fa0ae64030f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:31.663671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:31.745714 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:31.747862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1457/2916] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:31.759355 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.761219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:31.826431 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211902b-99bf-4997-9571-8ceb31e5eb7e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.830553 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.831251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.831639 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.831988 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:31.845884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:31.847601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:31.858724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1460/2917] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:31.940441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:31.950732 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f7991-068e-4aef-9065-79c1012bd301 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:31.954533 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:31.955147 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:31.955693 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:31.955969 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:32.015366 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:32.016808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fd66e-be55-4068-96a2-f0a1f2e07622 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:32.026288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1458/2918] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:36.115118 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:36.178706 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1657c6-038e-4fbf-a3ee-dc15cb658297 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:36.184251 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:36.184888 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:36.185201 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:36.185477 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] PATH_INFO: `/v3/users/978781e3bd4b4b6bb9bb2814d3992600` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:36.186842 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] RBAC: Authorizing `identity:delete_user(user_id=978781e3bd4b4b6bb9bb2814d3992600)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:36.188752 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:36.205233 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:36.239478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '978781e3bd4b4b6bb9bb2814d3992600'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.252420 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '978781e3bd4b4b6bb9bb2814d3992600'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.253175 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '978781e3bd4b4b6bb9bb2814d3992600'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.261900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '978781e3bd4b4b6bb9bb2814d3992600'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.269833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '978781e3bd4b4b6bb9bb2814d3992600'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.275906 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1461/2919] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:36 2022] DELETE /identity/v3/users/978781e3bd4b4b6bb9bb2814d3992600 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:36.280764 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:36.281246 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:36.281361 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:36.301125 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:36.302341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:36.363658 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1459/2920] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:36.373674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:36.375091 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:36.434343 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ee766-70d1-4c86-8a46-1e10e158718a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:36.438046 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:36.438666 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:36.438947 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:36.439225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:36.448368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:36.449847 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:36.459779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1462/2921] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:36.735264 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:36.736734 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:36.749663 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6f46-1f3e-4fed-b66d-c415e81eee88 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:36.755124 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:36.755807 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:36.756093 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:36.756378 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] PATH_INFO: `/v3/projects/909231165dcb42dcbdc397f7447945a5` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:36.757615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=909231165dcb42dcbdc397f7447945a5)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:36.760470 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:36.811389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '909231165dcb42dcbdc397f7447945a5', 'request_id': 'req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.817341 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '909231165dcb42dcbdc397f7447945a5', 'request_id': 'req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.823931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '909231165dcb42dcbdc397f7447945a5', 'request_id': 'req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.836744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] The token cache is being invalidate because project 909231165dcb42dcbdc397f7447945a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:36.837067 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.837691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.839710 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1460/2922] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:36 2022] DELETE /identity/v3/projects/909231165dcb42dcbdc397f7447945a5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:36.843744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:36.854967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101dc93-7f34-4c1e-8eb3-44a69f0689dd service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:36.911376 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:36.912021 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:36.912307 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:36.912589 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] PATH_INFO: `/v3/users/9ef8c1d283ee41af9c1fa0ae64030f1b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:36.913789 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef8c1d283ee41af9c1fa0ae64030f1b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:36.915719 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:36.957826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:36.964375 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef8c1d283ee41af9c1fa0ae64030f1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.978366 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef8c1d283ee41af9c1fa0ae64030f1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.979570 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef8c1d283ee41af9c1fa0ae64030f1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.986779 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef8c1d283ee41af9c1fa0ae64030f1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:36.994329 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ab5751e0-0eb3-4725-8a43-11e5cd777d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef8c1d283ee41af9c1fa0ae64030f1b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.000813 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1463/2923] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:36 2022] DELETE /identity/v3/users/9ef8c1d283ee41af9c1fa0ae64030f1b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:37.289709 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.291159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:37.306375 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f1799-a8b4-44b7-9c9c-87770ee243a7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.328464 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:37.371360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.372043 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.372324 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.372604 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] PATH_INFO: `/v3/projects/dac4e8e33fec464aa578a6cfb110781d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.373806 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac4e8e33fec464aa578a6cfb110781d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.375721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.422360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac4e8e33fec464aa578a6cfb110781d', 'request_id': 'req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.432274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac4e8e33fec464aa578a6cfb110781d', 'request_id': 'req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.438683 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac4e8e33fec464aa578a6cfb110781d', 'request_id': 'req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.452092 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] The token cache is being invalidate because project dac4e8e33fec464aa578a6cfb110781d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:37.452388 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.453051 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-853b95a5-27d8-4dfb-bb90-c1c7c15cb0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:37.454791 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1461/2924] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:37 2022] DELETE /identity/v3/projects/dac4e8e33fec464aa578a6cfb110781d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:37.460105 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.460559 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.460649 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.482213 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:37.483333 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:37.547100 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1464/2925] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:37.555576 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.555862 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.556170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.577557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:37.579038 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:37.587155 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1462/2926] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 20:54:37.591541 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.603918 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8fa70-84c3-4857-82b0-18936bb74354 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.610293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.610919 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.611230 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.611505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.612227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 20:54:37.612532 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.614167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.626577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1465/2927] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Jul 27 20:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 20:54:37.631072 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.642243 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a13d5e-e570-44d1-8c2c-b4680b6fe8a3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.646934 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.647556 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.647842 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.648119 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.648781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.650744 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.685668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1463/2928] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:37 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:37.685954 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.696082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c91f3-e413-4fb9-9537-7f657fcfe1d6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.756608 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.757168 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.757450 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.757724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.758377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.760288 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.781707 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1466/2929] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:37 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:37.785878 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.796275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcba85-44a4-4c00-9043-6ec9c9a60703 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.800892 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.801536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.801816 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.802088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.802776 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.804379 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.815310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1464/2930] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:37.825977 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.836241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284923c1-326b-49cb-8872-41b0dbfdb8e1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.841068 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.841672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.841949 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.842220 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.842899 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.844482 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39beb4c-1f57-4735-bb9c-985637935f55 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.857836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1467/2931] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:37.860539 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:37.871381 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b4f3a-b7fc-4f78-9123-c475657bc413 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:37.875993 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:37.876685 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.876979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.877286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] PATH_INFO: `/v3/projects/141408c4c75f4d6d951952d92dc927ac/users/0ae6f47a6c894a5da1280fcc4301ecb7/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.892242 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141408c4c75f4d6d951952d92dc927ac, user_id=0ae6f47a6c894a5da1280fcc4301ecb7, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:37.894967 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:37.908270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1465/2932] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:37 2022] PUT /identity/v3/projects/141408c4c75f4d6d951952d92dc927ac/users/0ae6f47a6c894a5da1280fcc4301ecb7/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:37.919247 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:37.919538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:37.919819 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:37.952029 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] MFA Rules not processed for user `0ae6f47a6c894a5da1280fcc4301ecb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:37.955255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:37.969549 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:38.028577 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1468/2933] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:38.033220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.084762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:38.100061 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7160eba-5a71-4260-b061-4c2b0c421547 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.105545 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.106165 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.106445 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.106716 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.107377 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.109320 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.137536 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1466/2934] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:38.141747 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.154535 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b734c3-6f21-4e61-a70f-bc6a8b553732 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.212581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.213227 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.214138 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.214432 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.215101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.217293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.238890 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1469/2935] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:38.242919 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.253701 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf7077-5e58-4736-be9f-bac0b89b756d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.258592 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.259220 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.259504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.259781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.261078 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.262808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.273827 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1467/2936] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:38.278741 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.288945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bc154-f430-4f1b-aac8-a740423ef68e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.294967 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.295581 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.295860 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.296133 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.296827 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.298475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c627b-a666-4996-800a-bb55876caf6a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.309440 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1470/2937] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:38.313843 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.324703 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29cef-30ab-4bc4-be80-4d8e886bd67a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.329643 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.330698 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.331002 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.331282 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] PATH_INFO: `/v3/projects/0baccf0a92534e7eb693775b54a339b4/users/9020936da85c46e5b96ac80fa5e75469/roles/cfc89a51451c403880880749d727c042` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.349207 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] RBAC: Authorizing `identity:create_grant(project_id=0baccf0a92534e7eb693775b54a339b4, user_id=9020936da85c46e5b96ac80fa5e75469, role_id=cfc89a51451c403880880749d727c042)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.352288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.365724 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1468/2938] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:38 2022] PUT /identity/v3/projects/0baccf0a92534e7eb693775b54a339b4/users/9020936da85c46e5b96ac80fa5e75469/roles/cfc89a51451c403880880749d727c042 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:38.377462 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.377750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.378031 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.406880 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] MFA Rules not processed for user `9020936da85c46e5b96ac80fa5e75469`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:38.408115 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:38.472715 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1471/2939] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:38.477485 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.541250 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586bf65-1e98-46c9-8a14-e622dfb024de admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.546760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.547374 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.547679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.547958 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.548629 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.550628 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.584369 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1469/2940] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 20:54:38.589013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.600338 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe5b2-d58b-4dcd-83a3-5f18f13ffac8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.617347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:38.674641 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.675370 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.675700 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.676020 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.676775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.678855 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.700417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1472/2941] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 20:54:38.704400 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.715923 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4f33a-fc0d-474c-9aaf-ff6ffb23391d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.720631 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.721286 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.721565 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.721870 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.722560 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.724158 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.737006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1470/2942] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:38.740660 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.751546 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881a3ae-a858-4e0f-8a9b-45164fd5753f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.757162 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.757854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.758134 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.758411 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] PATH_INFO: `/v3/projects/08c9f6895d724ecaba1ef10b9e99da00/users/d40cc9c1d653425c8e2dba0f2d8b8641/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.773746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c9f6895d724ecaba1ef10b9e99da00, user_id=d40cc9c1d653425c8e2dba0f2d8b8641, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.776469 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.791034 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1473/2943] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:38 2022] PUT /identity/v3/projects/08c9f6895d724ecaba1ef10b9e99da00/users/d40cc9c1d653425c8e2dba0f2d8b8641/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 20:54:38.795846 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.805838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d4bdf-37a5-4e81-a029-4f31a0acaa70 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.864760 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.865547 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.865838 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.866123 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.866841 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.868601 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb80ec-15c1-4be8-b233-05f4789cbd6d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.882931 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1471/2944] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Jul 27 20:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 20:54:38.887412 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:38.897651 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd57aa-b097-4839-839c-36ae78fd83ad admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:38.902139 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:38.902782 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.903059 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.903330 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] PATH_INFO: `/v3/projects/08c9f6895d724ecaba1ef10b9e99da00/users/d40cc9c1d653425c8e2dba0f2d8b8641/roles/656ff46e1a044f1a8f146570c85d81a0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.905403 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c9f6895d724ecaba1ef10b9e99da00, user_id=d40cc9c1d653425c8e2dba0f2d8b8641, role_id=656ff46e1a044f1a8f146570c85d81a0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:38.908082 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:38.919357 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1474/2945] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Jul 27 20:54:38 2022] PUT /identity/v3/projects/08c9f6895d724ecaba1ef10b9e99da00/users/d40cc9c1d653425c8e2dba0f2d8b8641/roles/656ff46e1a044f1a8f146570c85d81a0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:38.929435 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:38.929728 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:38.930006 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:38.951197 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:38.961774 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] MFA Rules not processed for user `d40cc9c1d653425c8e2dba0f2d8b8641`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:38.962991 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:39.042668 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1472/2946] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:39.054273 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:39.056150 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:39.116916 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570e650-71ef-4203-a845-88ea9a751ab5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:39.120869 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:39.121519 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:39.121795 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:39.122066 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:39.180399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:39.181885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:39.191652 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1475/2947] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:41.754526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:41.764591 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c825-5d72-47d3-890b-1043725e3842 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:41.768050 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:41.768651 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:41.768924 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:41.769215 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:41.825152 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:41.826503 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:41.835460 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1473/2948] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:42.354224 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:42.362994 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af4be3-b082-4068-92af-983e8b732eeb service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:42.366580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:42.367184 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:42.367465 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:42.367742 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:42.377206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:42.378616 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f926afe-7024-4757-b548-f2bb1f75685e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:42.388627 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1476/2949] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:46.253859 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:46.316088 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2934-d988-492b-907d-66d4466a7854 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:46.320567 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:46.321228 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:46.321510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:46.321786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] PATH_INFO: `/v3/users/0ae6f47a6c894a5da1280fcc4301ecb7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:46.322943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae6f47a6c894a5da1280fcc4301ecb7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:46.325424 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:46.363065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:46.373281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae6f47a6c894a5da1280fcc4301ecb7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.380042 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae6f47a6c894a5da1280fcc4301ecb7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.392587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae6f47a6c894a5da1280fcc4301ecb7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.399345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae6f47a6c894a5da1280fcc4301ecb7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.399988 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae6f47a6c894a5da1280fcc4301ecb7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.406437 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1474/2950] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:46 2022] DELETE /identity/v3/users/0ae6f47a6c894a5da1280fcc4301ecb7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:46.410884 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:46.411342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:46.411454 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:46.429833 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] MFA Rules not processed for user `8e99a25baa6a4ee4875a0277c6006aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:46.431023 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:46.438938 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:46.491692 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1477/2951] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:46.501786 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:46.503201 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:46.559860 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638eb3-d7a8-43a2-b014-a1a99499530b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:46.563457 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:46.564064 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:46.564340 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:46.564609 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:46.573965 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:46.575530 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:46.585022 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1475/2952] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:54:46.849765 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:46.851321 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:46.865342 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddaabd-6451-4564-a013-f8466823cf51 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:46.870979 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:46.871743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:46.872083 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:46.872362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] PATH_INFO: `/v3/projects/141408c4c75f4d6d951952d92dc927ac` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:46.873566 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] RBAC: Authorizing `identity:delete_project(project_id=141408c4c75f4d6d951952d92dc927ac)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:46.876206 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:46.923493 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141408c4c75f4d6d951952d92dc927ac', 'request_id': 'req-bd189ece-8a29-4a58-b7a2-66e112ff687b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.928800 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141408c4c75f4d6d951952d92dc927ac', 'request_id': 'req-bd189ece-8a29-4a58-b7a2-66e112ff687b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.935368 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141408c4c75f4d6d951952d92dc927ac', 'request_id': 'req-bd189ece-8a29-4a58-b7a2-66e112ff687b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.948362 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] The token cache is being invalidate because project 141408c4c75f4d6d951952d92dc927ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:46.948671 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.949295 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:46.950775 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1478/2953] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:46 2022] DELETE /identity/v3/projects/141408c4c75f4d6d951952d92dc927ac => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:46.954866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:46.964513 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7eaf-856e-49a9-ad98-14ebdaba75c0 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:47.019891 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:47.020682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:47.020840 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:47.021094 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] PATH_INFO: `/v3/users/9020936da85c46e5b96ac80fa5e75469` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:47.022245 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9020936da85c46e5b96ac80fa5e75469)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:47.024159 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:47.068236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9020936da85c46e5b96ac80fa5e75469'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.076232 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9020936da85c46e5b96ac80fa5e75469'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.079403 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:47.093515 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9020936da85c46e5b96ac80fa5e75469'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.100430 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9020936da85c46e5b96ac80fa5e75469'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.101171 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9020936da85c46e5b96ac80fa5e75469'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.106995 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1476/2954] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:46 2022] DELETE /identity/v3/users/9020936da85c46e5b96ac80fa5e75469 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:47.388274 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:47.389744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:47.404407 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189ece-8a29-4a58-b7a2-66e112ff687b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:47.463377 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:47.464041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:47.464237 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:47.464509 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] PATH_INFO: `/v3/projects/0baccf0a92534e7eb693775b54a339b4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:47.465682 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0baccf0a92534e7eb693775b54a339b4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:47.467575 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:47.506478 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:47.515238 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0baccf0a92534e7eb693775b54a339b4', 'request_id': 'req-36293e89-6f63-46d5-9812-2c03a10e1e05'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.520599 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0baccf0a92534e7eb693775b54a339b4', 'request_id': 'req-36293e89-6f63-46d5-9812-2c03a10e1e05'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.526588 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0baccf0a92534e7eb693775b54a339b4', 'request_id': 'req-36293e89-6f63-46d5-9812-2c03a10e1e05'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.538417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] The token cache is being invalidate because project 0baccf0a92534e7eb693775b54a339b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:47.538705 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.539322 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.540803 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1479/2955] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:47 2022] DELETE /identity/v3/projects/0baccf0a92534e7eb693775b54a339b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:47.544787 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:47.546310 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:47.559824 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff3fc2-7005-4e04-8326-f58825bb3962 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:47.615638 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:47.616272 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:47.616554 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:47.616828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] PATH_INFO: `/v3/users/d40cc9c1d653425c8e2dba0f2d8b8641` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:47.618004 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d40cc9c1d653425c8e2dba0f2d8b8641)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:47.620427 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:47.665474 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd40cc9c1d653425c8e2dba0f2d8b8641'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.673288 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd40cc9c1d653425c8e2dba0f2d8b8641'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.686082 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd40cc9c1d653425c8e2dba0f2d8b8641'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.692813 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd40cc9c1d653425c8e2dba0f2d8b8641'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.693478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-acdcb837-4ced-49c3-9009-f5fecf9f732e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd40cc9c1d653425c8e2dba0f2d8b8641'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:47.699393 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1477/2956] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 20:54:47 2022] DELETE /identity/v3/users/d40cc9c1d653425c8e2dba0f2d8b8641 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:47.969128 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:47.970567 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:47.985241 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36293e89-6f63-46d5-9812-2c03a10e1e05 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:48.042753 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:48.043489 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:48.043774 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:48.044056 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] PATH_INFO: `/v3/projects/08c9f6895d724ecaba1ef10b9e99da00` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:48.045236 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c9f6895d724ecaba1ef10b9e99da00)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:48.047154 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:48.092945 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c9f6895d724ecaba1ef10b9e99da00', 'request_id': 'req-f133953a-a2ca-491c-90da-e40ad56742a8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:48.098475 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c9f6895d724ecaba1ef10b9e99da00', 'request_id': 'req-f133953a-a2ca-491c-90da-e40ad56742a8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:48.105239 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c9f6895d724ecaba1ef10b9e99da00', 'request_id': 'req-f133953a-a2ca-491c-90da-e40ad56742a8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:48.117430 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] The token cache is being invalidate because project 08c9f6895d724ecaba1ef10b9e99da00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 20:54:48.117720 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:48.118369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 20:54:48.119900 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1480/2957] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 20:54:47 2022] DELETE /identity/v3/projects/08c9f6895d724ecaba1ef10b9e99da00 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 20:54:57.695739 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=63,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:57.800572 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:57.800861 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:57.801180 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:57.825270 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] MFA Rules not processed for user `060f4e9543a5456897d5217676cb93a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:54:57.826478 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:57.881900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1478/2958] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:54:58.114373 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:54:58.199876 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:54:58.201419 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:58.257190 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133953a-a2ca-491c-90da-e40ad56742a8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:54:58.261361 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:54:58.261971 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:54:58.262252 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:54:58.262527 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:54:58.264263 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:54:58.273525 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:54:58.274933 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:54:58.283293 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1481/2959] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:55:07.875080 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=25 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:55:08.279762 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=26 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:55:21.966587 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:55:21.976101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97f760-17bf-4aed-b8ff-d7060d805a3f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:55:21.979620 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:55:21.980277 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:55:21.980575 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:55:21.980850 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:55:21.982597 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:55:22.047052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:55:22.048490 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:55:22.057961 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1479/2960] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:55:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:55:32.053781 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=30 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:56:54.602460 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:56:54.611186 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641714e5-a361-4f20-a20e-126d85189f51 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:56:54.614927 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:56:54.615531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:56:54.615810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:56:54.616088 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:56:54.617862 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:56:54.640041 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:56:54.641499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:56:54.650123 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1482/2961] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:04.646473 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:57:37.702653 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:37.704192 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:37.716322 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af1dbc-2395-4d92-b745-82e84f7a3569 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:37.720526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:37.721992 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:37.722276 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:37.722557 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:37.724308 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:37.812287 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:37.813672 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:37.825034 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1480/2962] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:39.004816 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:39.013646 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d96fc-9867-42de-ba57-671c50d38a13 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:39.017253 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:39.017852 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:39.018129 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:39.018402 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:39.020245 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:39.038531 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:39.039956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a91a4e-ee17-4ba2-8d21-05a73660dcb3 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:39.051561 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1483/2963] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:40.342359 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:40.343866 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:40.358874 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d14fe-e4d0-4ec4-9703-b5f1889028de service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:40.365423 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:40.366052 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:40.366336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:40.366615 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:40.368856 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:40.384721 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:40.386713 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:40.400127 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1481/2964] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:41.335808 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17512cf-6275-4872-87b4-608af785678a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:41.336399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17512cf-6275-4872-87b4-608af785678a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:41.336399 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17512cf-6275-4872-87b4-608af785678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:41.355750 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d17512cf-6275-4872-87b4-608af785678a None None] MFA Rules not processed for user `060f4e9543a5456897d5217676cb93a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 20:57:41.356956 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d17512cf-6275-4872-87b4-608af785678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:41.364738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1484/2965] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 20:57:42.133678 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:42.135194 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:42.208657 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50c541-1f36-4a52-9c33-6440fa72861e service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:42.213920 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:42.214526 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:42.214808 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:42.215087 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:42.216887 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:42.228189 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:42.229796 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:42.240356 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1482/2966] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:51.350388 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:57:52.235360 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=77 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:57:55.543557 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17512cf-6275-4872-87b4-608af785678a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:55.579427 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17512cf-6275-4872-87b4-608af785678a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:55.585743 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:55.586369 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:55.586655 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:55.586936 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:55.588769 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:55.613673 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:55.615272 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:55.626447 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1485/2967] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:57:58.867426 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:57:58.916762 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808dffa-113c-4afc-bb19-ea9027bb57ef service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:57:58.922065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:57:58.922691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:57:58.922979 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:57:58.923261 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:57:58.925682 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:57:58.950812 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:57:58.952406 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84a64-241c-425d-ac57-037190076649 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:57:58.963170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1483/2968] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:58:05.621485 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=19 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:58:08.958635 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=27 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:59:48.883717 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 20:59:48.885182 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:59:48.940858 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f98395-a361-4460-989b-53d214cb3acf service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 20:59:48.944672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5c57fe9a7f141458ac3d01bfa045695', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 20:59:48.945317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:59:48.945601 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:59:48.945873 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 20:59:48.947580 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 20:59:48.958179 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 20:59:48.959716 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8118f1-4a11-4f32-9862-3d548ae078b6 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 20:59:48.968773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1486/2969] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 20:59:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 20:59:58.964678 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=26 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 20:59:59.979115 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 20:59:59.979690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 20:59:59.979690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:00.000336 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] MFA Rules not processed for user `060f4e9543a5456897d5217676cb93a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:00:00.001603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:00:00.009441 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1484/2970] 10.222.0.65 () {58 vars in 958 bytes} [Wed Jul 27 20:59:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:00:00.702300 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558be46-ff3e-4f05-b706-1600032cf6a3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:00:00.702854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558be46-ff3e-4f05-b706-1600032cf6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:00:00.702854 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558be46-ff3e-4f05-b706-1600032cf6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:00.759417 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d558be46-ff3e-4f05-b706-1600032cf6a3 None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:00:00.760605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d558be46-ff3e-4f05-b706-1600032cf6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:00:00.767013 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1487/2971] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Jul 27 21:00:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:00:00.773440 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:00:00.782773 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698ff8e-9034-490a-b2c0-07bd7df88d19 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:00:00.786300 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:00:00.787600 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:00:00.787879 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:00:00.788161 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:00.795606 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:00:00.797101 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:00:00.805330 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1485/2972] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:00:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 21:00:03.262794 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:00:03.263231 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:00:03.263347 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:03.301062 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:00:03.302946 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:00:03.311511 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1488/2973] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Jul 27 21:00:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:00:03.323633 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:00:03.325065 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:00:03.341009 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caa565-94c4-4be5-8219-4005ccb0025b service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:00:03.345998 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:00:03.346603 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:00:03.346994 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:00:03.347274 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:03.356671 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:00:03.358397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:00:03.368987 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1486/2974] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:00:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:00:13.298414 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=26 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:00:13.364074 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=15 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:00:23.227745 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:00:23.236773 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f797-57da-4b2c-b056-0fb1dd99dd79 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:00:23.240311 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:00:23.240915 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:00:23.241225 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:00:23.241512 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:00:23.250336 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:00:23.251771 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:00:23.261620 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1489/2975] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:00:33.256877 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:02:00.607857 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:00.638027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbdac6-4545-4ddc-8193-6ee07a63b3a6 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:00.641661 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:00.642268 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:00.642566 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:00.642828 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:00.650110 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:00.651548 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5227fc8-47f7-4e3c-bf38-35feecd30f0d service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:00.659543 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1487/2976] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:02:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:02:10.655943 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=15 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:02:21.666744 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:21.668810 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:21.726756 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bc431-215d-4538-ba98-33942bedf38e service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:21.733632 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '056e9b8fcfc84ee68a018f331e805582', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:21.734260 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:21.734543 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:21.734820 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:21.736674 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:21.750836 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:21.752481 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:21.765624 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1490/2977] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:02:27.153281 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:27.153746 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:27.153877 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:27.175239 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] MFA Rules not processed for user `6e73fb42abfa4d61b610555f657ec062`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:02:27.176431 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:27.182948 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1488/2978] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Jul 27 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:02:27.188746 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:27.198283 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b880-5dee-4897-9037-e4258cd8bdb8 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:27.202072 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e73fb42abfa4d61b610555f657ec062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:27.202703 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:27.203015 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:27.203317 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:27.205159 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:27.288199 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:27.289605 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:27.308724 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1491/2979] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:02:30.345397 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:30.358873 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53528fe-90f0-40f2-aad8-c99fd00f7cdf None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:30.365275 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:30.366135 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:30.366476 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:30.366759 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:30.368690 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:30.381928 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:30.383553 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:30.396170 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1489/2980] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:02:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:02:33.583910 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:33.585598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:33.598550 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afabb27-52bd-4d69-85c7-864b7f090253 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:33.603167 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:33.603812 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:33.604101 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:33.604380 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] PATH_INFO: `/v3/users/060f4e9543a5456897d5217676cb93a6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:33.606255 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060f4e9543a5456897d5217676cb93a6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:33.608170 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:33.655192 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060f4e9543a5456897d5217676cb93a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.667009 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060f4e9543a5456897d5217676cb93a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.667695 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060f4e9543a5456897d5217676cb93a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.672853 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:02:33.674874 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060f4e9543a5456897d5217676cb93a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.683568 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060f4e9543a5456897d5217676cb93a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.689823 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1492/2981] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 21:02:33 2022] DELETE /identity/v3/users/060f4e9543a5456897d5217676cb93a6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 21:02:33.863389 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:33.874504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e5e54-a675-405b-a3dd-c6ae3aecfe88 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:33.930849 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:33.931415 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:33.931691 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:33.931964 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] PATH_INFO: `/v3/projects/c51423871031456a80da2d9d0c74bc62` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:33.933346 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51423871031456a80da2d9d0c74bc62)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:33.935226 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:33.982568 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51423871031456a80da2d9d0c74bc62', 'request_id': 'req-dfca47d7-57d0-4bfb-be85-71ffe582fe70'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.988236 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51423871031456a80da2d9d0c74bc62', 'request_id': 'req-dfca47d7-57d0-4bfb-be85-71ffe582fe70'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:33.994608 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51423871031456a80da2d9d0c74bc62', 'request_id': 'req-dfca47d7-57d0-4bfb-be85-71ffe582fe70'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.006679 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] The token cache is being invalidate because project c51423871031456a80da2d9d0c74bc62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 21:02:34.006960 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.007582 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.009136 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1490/2982] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 21:02:33 2022] DELETE /identity/v3/projects/c51423871031456a80da2d9d0c74bc62 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 21:02:34.013538 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:34.014955 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:34.028672 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6174836-1153-4bc9-8bc7-0be1f0ae9989 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:34.084085 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:34.084729 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:34.085043 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:34.085325 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] PATH_INFO: `/v3/users/51830aea6d6040f6b0b09e13ea95dfa4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:34.086499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51830aea6d6040f6b0b09e13ea95dfa4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:34.088437 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:34.133613 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51830aea6d6040f6b0b09e13ea95dfa4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.146505 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51830aea6d6040f6b0b09e13ea95dfa4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.147176 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51830aea6d6040f6b0b09e13ea95dfa4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.154598 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51830aea6d6040f6b0b09e13ea95dfa4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.161885 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51830aea6d6040f6b0b09e13ea95dfa4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.167838 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1493/2983] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Jul 27 21:02:34 2022] DELETE /identity/v3/users/51830aea6d6040f6b0b09e13ea95dfa4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 21:02:34.336792 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:02:34.338240 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:02:34.353504 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca47d7-57d0-4bfb-be85-71ffe582fe70 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:02:34.411257 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e99a25baa6a4ee4875a0277c6006aec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e55cde5d6044829b509ee6bccad61b8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:02:34.411900 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:02:34.412211 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:02:34.412486 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] PATH_INFO: `/v3/projects/304318c87f724855bbccde4c0fc414e0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:02:34.413684 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=304318c87f724855bbccde4c0fc414e0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:02:34.415664 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:02:34.426886 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:02:34.461492 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304318c87f724855bbccde4c0fc414e0', 'request_id': 'req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.467510 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304318c87f724855bbccde4c0fc414e0', 'request_id': 'req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.473875 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304318c87f724855bbccde4c0fc414e0', 'request_id': 'req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.488070 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] The token cache is being invalidate because project 304318c87f724855bbccde4c0fc414e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 21:02:34.488363 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.489039 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b4c02308-c41f-4ed7-a0ba-d7941fcdc1bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 21:02:34.490834 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1491/2984] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Jul 27 21:02:34 2022] DELETE /identity/v3/projects/304318c87f724855bbccde4c0fc414e0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 21:02:44.164298 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:02:44.484809 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:03:04.860786 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:03:04.862242 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:03:04.929736 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3eeaa6-6670-46b0-b330-5e5e48fa1c90 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:03:04.988760 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:03:04.989426 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:03:04.989738 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:03:04.990016 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:03:04.991826 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:03:05.003442 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:03:05.005045 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d115d2-7de1-4ff8-ae05-1f6e07c597a9 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:03:05.015409 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1494/2985] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:03:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:03:15.011216 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=58 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:05:04.888914 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:05:04.889529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:05:04.889529 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:05:04.909885 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:05:04.911104 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:05:04.919903 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1492/2986] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:05:04.932499 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c073716-de86-4522-92c4-3db43b15e2c9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:05:04.932944 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c073716-de86-4522-92c4-3db43b15e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:05:04.933078 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c073716-de86-4522-92c4-3db43b15e2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:05:04.952479 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0c073716-de86-4522-92c4-3db43b15e2c9 None None] MFA Rules not processed for user `6d75086318b148b59d70031805acdc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 21:05:04.953712 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c073716-de86-4522-92c4-3db43b15e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 21:05:04.962814 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1495/2987] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Jul 27 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 21:05:04.968819 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 21:05:04.979302 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bb897-3a12-4579-857d-5da4968e9b19 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 21:05:04.984367 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d75086318b148b59d70031805acdc9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9e9eccd156dd4ed69e5b6dad01362098', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 21:05:04.985027 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 21:05:04.985314 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 21:05:04.985589 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 21:05:04.994157 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 21:05:04.995737 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a1e00-44fb-4da8-9d1e-b01fcbf4bfb8 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 21:05:05.005894 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1493/2988] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Jul 27 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 21:05:14.947219 dv-cinder-799871-44-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 21:05:15.001345 dv-cinder-799871-44-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=9 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}